00010000:00010000:3.0:1787784569.372987:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373009:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373017:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373024:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373114:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905600/0xb0ce63137acc3ae6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373123:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905600/0xb0ce63137acc3ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373129:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905600/0xb0ce63137acc3ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784569.373139:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1e:0x0]: returning size 1516 00010000:00010000:3.0:1787784569.373615:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373624:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373630:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.373636:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784569.374242:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374340:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374356:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff989531111a00) 00010000:00010000:3.0:1787784569.374359:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784569.374364:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374380:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3ad8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374397:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374403:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374409:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374415:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784569.374418:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374424:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374430:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x193a76c9 (ffff989510520800) 00010000:00010000:3.0:1787784569.374433:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784569.374436:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374448:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3adf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374522:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc3ad1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03e9 expref: 68 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374537:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905600/0xb0ce63137acc3ae6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03e9 expref: 67 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784569.374544:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff989519d44300) 00010000:00010000:3.0:1787784569.374546:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784569.374549:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905600, rc 0) 00000100:00100000:3.0:1787784569.374661:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:15366:x1874627657912960:12345-192.168.203.43@tcp:101:multiop.0 Request processed in 1925us (2198us total) trans 4294967667 rc 0/0 00000100:00100000:3.0:1787784569.374669:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 483 00010000:00010000:3.0:1787784570.801509:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784572.305455:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967326 is committed 00000100:00100000:3.0:1787784572.318581:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657914752 00000100:00100000:3.0:1787784572.318587:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657914752 00002000:00100000:3.0:1787784572.318619:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xa:0x0] (0->1515) req@ffff98951b535680 x1874627657914752/t0(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:453/0 lens 2008/0 e 0 to 0 dl 1787784583 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784572.318631:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b535680 with x1874627657914752 ext(0->1515) 00002000:00010000:3.0:1787784572.318637:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b535680 00000100:00100000:3.0:1787784572.318653:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 48 00000100:00100000:3.0:1787784572.318661:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627657914752:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:3.0:1787784572.323028:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2355:x1874627657914752:12345-192.168.203.43@tcp:4:cp.0 Request processed in 4350us (5157us total) trans 4294967322 rc 0/0 00000100:00100000:3.0:1787784572.323057:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 48 00002000:00100000:3.0:1787784572.323088:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xa:0x0] (0->1515) req@ffff98951b535680 x1874627657914752/t4294967322(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:453/0 lens 2008/448 e 0 to 0 dl 1787784583 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784572.323099:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b535680 with x1874627657914752 ext(0->1515) 00002000:00010000:3.0:1787784572.323107:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b535680 00000100:00080000:3.0:1787784572.685261:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784572.686208:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657915008 00000100:00100000:3.0:1787784572.686218:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657915008 00000100:00100000:3.0:1787784572.686279:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 313 00000100:00100000:3.0:1787784572.686291:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627657915008:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784572.686359:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627657915008:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 70us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784572.686368:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 313 00000100:00080000:3.0:1787784572.687715:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784572.688274:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784572.688279:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784572.852941:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784574.450833:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668312960 00000100:00100000:3.0:1787784574.450839:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668312960 00000100:00100000:3.0:1787784574.450877:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:3.0:1787784574.450887:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668312960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784574.450953:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668312960:12345-0@lo:400:kworker.0 Request processed in 68us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784574.450959:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:3.0:1787784574.450985:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668312832 00000100:00100000:3.0:1787784574.450989:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668312832 00000100:00100000:3.0:1787784574.451033:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1787784574.451039:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668312832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784574.451080:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668312832:12345-0@lo:400:kworker.0 Request processed in 41us (218us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784574.451086:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1787784574.451133:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668312320 00000100:00100000:3.0:1787784574.451137:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668312320 00000100:00100000:3.0:1787784574.451182:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:3.0:1787784574.451191:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668312320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784574.451222:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668312320:12345-0@lo:400:kworker.0 Request processed in 34us (910us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784574.451229:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00080000:3.0:1787784574.451271:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784574.451313:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668312832:0@lo:400:kworker.0 00000100:00100000:3.0:1787784574.452710:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668312320:0@lo:400:kworker.0 00000100:00080000:3.0:1787784575.436238:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784575.436397:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657916160 00000100:00100000:3.0:1787784575.436414:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657916160 00000100:00100000:3.0:1787784575.436495:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 487 00000100:00100000:3.0:1787784575.436507:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+16:15935:x1874627657916160:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:3.0:1787784575.436586:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784575.436623:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3b17 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03f7 expref: 16 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.436647:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989531111f00) 00010000:00010000:3.0:1787784575.436653:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784575.437090:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3b1e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.437110:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc35fa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b028b expref: 45 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.437120:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3b1e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.437211:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc35fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b028b expref: 45 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.437230:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc35fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b028b expref: 45 pid: 10850 timeout: 466 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784575.437248:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09da40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668313216:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784575.437379:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895012edb40 going to sleep for 11 seconds 00010000:00010000:3.0:1787784575.446944:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc35fa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b028b expref: 46 pid: 10850 timeout: 466 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784575.446973:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09da40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668313216:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784575.447015:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3b1e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.447029:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3b1e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.447036:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3b1e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.447093:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc3b25 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.447108:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc3b25 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.447119:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc3b25 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784575.486565:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:3.0:1787784575.530180:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784575.563476:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784575.709619:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+18:15944:x1874627657917440:12345-192.168.203.43@tcp:44:dd.0 Request processed in 108248us (108905us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784575.709633:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 493 00000100:00100000:3.0:1787784575.711944:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784575.817528:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657917824 00000100:00100000:3.0:1787784575.817552:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657917824 00000100:00100000:3.0:1787784575.817626:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 114 00000100:00100000:3.0:1787784575.817638:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+18:15944:x1874627657917824:12345-192.168.203.43@tcp:35:dd.0 00000100:00100000:3.0:1787784575.822972:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+19:15944:x1874627657917824:12345-192.168.203.43@tcp:35:dd.0 Request processed in 5314us (5616us total) trans 4294967672 rc 0/0 00000100:00100000:3.0:1787784575.822992:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 114 00000100:00100000:3.0:1787784575.907287:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784575.935456:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:3.0:1787784575.963446:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784575.970922:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657918464 00000100:00100000:3.0:1787784575.970967:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657918464 00000100:00100000:3.0:1787784575.971033:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 497 00000100:00100000:3.0:1787784575.971046:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+48:15950:x1874627657918464:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784575.971090:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784575.971121:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3bf7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0444 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.971261:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3bfe lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.971279:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3bfe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.971294:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3bfe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784575.972373:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa:0x0]: returning size 7680 00000004:00200000:3.0:1787784575.972384:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784575.972529:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3bfe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.973001:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3bf7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0444 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.973028:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904000/0xb0ce63137acc3bfe lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0444 expref: 49 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784575.973038:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x0 (ffff98950a649e00) 00010000:00010000:3.0:1787784575.973041:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784575.973045:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904000, rc 0) 00000100:00100000:3.0:1787784575.973442:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+49:15950:x1874627657918464:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2395us (2624us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784575.973452:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 497 00000040:00080000:3.0:1787784576.177692:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784576.177703:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784576.178117:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784576.178129:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1787784576.786840:0:14463:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784576.786849:0:14463:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784576.788017:0:14463:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784576.788033:0:14463:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784576.948755:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784577.972713:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784579.571845:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668313984:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.571902:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784579.571962:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787784579.571968:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668313856:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.571996:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784579.572017:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668314368:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.572052:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784579.572077:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668314240:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.572107:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784579.572127:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784579.572132:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668314112:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.572175:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784579.572212:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668313856 00000100:00100000:3.0:1787784579.572218:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668313856 00000100:00100000:3.0:1787784579.572263:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:3.0:1787784579.572270:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668313856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.572315:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668313856:12345-0@lo:400:kworker.0 Request processed in 45us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784579.572322:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:3.0:1787784579.572350:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668314240 00000100:00100000:3.0:1787784579.572354:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668314240 00000100:00100000:3.0:1787784579.573111:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:3.0:1787784579.573121:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668314240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.573170:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668314240:12345-0@lo:400:kworker.0 Request processed in 51us (1062us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784579.573175:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00080000:3.0:1787784579.573201:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784579.573204:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784579.573231:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668314112:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.573243:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668314240:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.573270:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668313856:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.573280:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668314368:0@lo:400:kworker.0 00000100:00100000:3.0:1787784579.573289:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668313984:0@lo:400:kworker.0 00000040:00080000:3.0:1787784580.273415:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784580.273425:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784580.275128:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784580.275721:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784582.642943:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668314496:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784582.643047:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1787784582.644617:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784582.930971:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657919872 00000100:00100000:3.0:1787784582.930977:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657919872 00000100:00100000:3.0:1787784582.931034:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 317 00000100:00100000:3.0:1787784582.931042:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657919872:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784582.931135:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657919872:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 94us (1506us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784582.931143:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 317 00010000:00010000:3.0:1787784584.210110:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc3c13 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210124:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc3c13 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210131:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc3c13 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210137:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.210143:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc3c13 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210188:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc3c13 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210205:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:3.0:1787784584.210208:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.210214:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc3c13 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210234:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc3c13 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210255:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17400/0xb0ce63137acc3c1a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210262:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17400/0xb0ce63137acc3c1a lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210268:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17400/0xb0ce63137acc3c1a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210273:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.210276:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17400/0xb0ce63137acc3c1a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210282:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17400/0xb0ce63137acc3c1a lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210290:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff98950a162a00) 00010000:00010000:3.0:1787784584.210293:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.210297:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17400/0xb0ce63137acc3c1a lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210302:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17400/0xb0ce63137acc3c1a lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210844:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc3c0c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0452 expref: 21 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210876:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc3c21 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0452 expref: 20 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.210885:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784584.210888:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.210890:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c17000, rc 0) 00000100:00100000:3.0:1787784584.210991:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16552:x1874627657920768:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2757us (3165us total) trans 4294967673 rc 0/0 00000100:00100000:3.0:1787784584.211014:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 499 00000100:00100000:3.0:1787784584.224678:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784584.229802:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657921152 00000100:00100000:3.0:1787784584.229815:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657921152 00000100:00100000:3.0:1787784584.229913:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00100000:3.0:1787784584.229924:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+49:16553:x1874627657921152:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784584.229951:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.229980:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c2f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0459 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.230076:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.230094:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.230107:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.230178:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.230187:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.230194:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232552:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232573:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232590:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232597:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.232603:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232612:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232633:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649500) 00010000:00010000:3.0:1787784584.232636:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.232641:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232649:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c36 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232968:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232975:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232981:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.232986:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.232989:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.233069:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.233078:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc1d08857 (ffff989510520700) 00010000:00010000:3.0:1787784584.233080:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.233084:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.233089:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c3d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.233171:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c2f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0459 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.233241:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649500) 00010000:00010000:3.0:1787784584.233245:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.233257:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c2f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0459 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.233268:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904e00, rc 0) 00000100:00100000:3.0:1787784584.233371:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+49:16553:x1874627657921152:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3448us (4085us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784584.233382:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00100000:3.0:1787784584.242228:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657921408 00000100:00100000:3.0:1787784584.242250:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657921408 00000100:00100000:3.0:1787784584.242338:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 502 00000100:00100000:3.0:1787784584.242350:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+49:16553:x1874627657921408:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:3.0:1787784584.242429:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784584.242469:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac400/0xb0ce63137acc3bb1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0428 expref: 49 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.242492:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649500) 00010000:00010000:3.0:1787784584.242499:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784584.243895:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c44 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.243927:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc3b33 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03fe expref: 18 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.243936:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c44 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.243990:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc3b33 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03fe expref: 18 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.244041:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc3b33 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03fe expref: 18 pid: 10850 timeout: 475 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784584.244060:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668314624:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784584.244225:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063efc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784584.246476:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668314624:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784584.246513:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c44 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784584.246823:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657921536 00000100:00100000:3.0:1787784584.246831:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657921536 00010000:00010000:3.0:1787784584.246879:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc3b33 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03fe expref: 19 pid: 10850 timeout: 475 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784584.246926:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 164 00000100:00100000:3.0:1787784584.246936:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2357:x1874627657921536:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784584.246956:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784584.246963:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc3b33 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03fe expref: 19 pid: 10850 timeout: 475 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.246981:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc3b33 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03fe expref: 19 pid: 10850 timeout: 475 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.247016:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc3b33 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b03fe expref: 19 pid: 10850 timeout: 475 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.247056:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649500) 00010000:00010000:3.0:1787784584.247062:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c44 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.247072:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c44 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.247081:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3c44 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.247280:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784584.247341:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2357:x1874627657921536:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 405us (729us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784584.247349:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 164 00000100:00100000:3.0:1787784584.269392:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657921920 00000100:00100000:3.0:1787784584.269405:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657921920 00000100:00100000:3.0:1787784584.269476:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 505 00000100:00100000:3.0:1787784584.269486:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+51:16554:x1874627657921920:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784584.269511:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.269539:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c7c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b046e expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269638:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269655:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269701:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269722:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269732:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269741:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269840:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269847:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269853:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269859:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.269864:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269872:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269887:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.269890:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.269896:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269909:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3c83 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269925:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269931:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269936:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269941:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.269944:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269950:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269957:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff989510520700) 00010000:00010000:3.0:1787784584.269959:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.269962:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.269967:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3c8a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.270137:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c7c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b046e expref: 52 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.270209:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.270212:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.270216:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c7c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b046e expref: 52 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.270227:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904c00, rc 0) 00000100:00100000:3.0:1787784584.270300:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+51:16554:x1874627657921920:12345-192.168.203.43@tcp:101:bash.0 Request processed in 815us (1015us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784584.270309:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 505 00000100:00100000:3.0:1787784584.290461:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657922176 00000100:00100000:3.0:1787784584.290473:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657922176 00000100:00100000:3.0:1787784584.290547:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 507 00000100:00100000:3.0:1787784584.290558:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+19:16555:x1874627657922176:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784584.290595:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.290624:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c9f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b047c expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290746:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290763:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290775:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290791:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290804:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290810:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290962:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3cb4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290970:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.290977:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784584.291025:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x20:0x0]: returning size 512 00000004:00200000:3.0:1787784584.291029:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784584.291057:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291070:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291077:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291084:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291091:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.291095:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291110:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291124:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649f00) 00010000:00010000:3.0:1787784584.291128:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.291133:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291140:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3ca6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291183:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291189:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291194:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291200:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.291203:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291209:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291216:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc1d08857 (ffff989510520700) 00010000:00010000:3.0:1787784584.291218:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.291221:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291238:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cad lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291275:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3c9f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b047c expref: 21 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291291:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3cb4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b047c expref: 20 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.291299:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.291301:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784584.291304:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904e00, rc 0) 00000100:00100000:3.0:1787784584.291387:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16555:x1874627657922176:12345-192.168.203.43@tcp:101:rm.0 Request processed in 831us (1282us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784584.291396:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 507 00000100:00100000:3.0:1787784584.316605:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657922432 00000100:00100000:3.0:1787784584.316619:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657922432 00000100:00100000:3.0:1787784584.316710:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00100000:3.0:1787784584.316722:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16555:x1874627657922432:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784584.316751:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.316779:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3cd0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b048a expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.316869:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.316887:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.316922:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.316939:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.316947:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.316953:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317052:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317062:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317069:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317076:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.317082:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317090:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317106:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.317110:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.317115:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317128:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cd7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317651:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317663:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317717:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317725:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.317729:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317737:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317747:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff989510520700) 00010000:00010000:3.0:1787784584.317750:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.317754:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317761:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cde lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317812:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3cd0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b048a expref: 21 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317841:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.317844:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.317848:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3cd0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b048a expref: 21 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.317860:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904c00, rc 0) 00000100:00100000:3.0:1787784584.317946:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16555:x1874627657922432:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1226us (1509us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784584.317956:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00100000:3.0:1787784584.333538:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657922560 00000100:00100000:3.0:1787784584.333551:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657922560 00000100:00100000:3.0:1787784584.333954:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 510 00000100:00100000:3.0:1787784584.333968:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16556:x1874627657922560:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784584.334010:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.334039:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3ce5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0491 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334128:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334193:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334209:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334226:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334253:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334259:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334347:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334356:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334363:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334369:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.334374:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334382:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334397:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.334401:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.334406:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334412:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc3cec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334428:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334441:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334448:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334454:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.334457:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334464:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334473:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff989510520700) 00010000:00010000:3.0:1787784584.334476:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.334479:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334486:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc3cf3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334524:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3ce5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0491 expref: 21 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334536:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.334538:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.334542:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc3ce5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0491 expref: 21 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.334574:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904c00, rc 0) 00000100:00100000:3.0:1787784584.334655:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16556:x1874627657922560:12345-192.168.203.43@tcp:101:cp.0 Request processed in 690us (1297us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784584.334664:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 510 00000100:00100000:3.0:1787784584.338801:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657922688 00000100:00100000:3.0:1787784584.338813:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657922688 00000100:00100000:3.0:1787784584.338886:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 511 00000100:00100000:3.0:1787784584.338897:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16556:x1874627657922688:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784584.338925:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.338952:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc3cfa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0498 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339054:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339071:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339085:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339102:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339110:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339116:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339253:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339261:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339268:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339274:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.339279:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339287:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339303:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.339306:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.339311:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339318:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc3d01 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339333:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339338:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339344:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339356:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.339360:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339366:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339375:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff989510520900) 00010000:00010000:3.0:1787784584.339377:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.339379:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339385:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d08 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339425:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc3cfa lrc: 1/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0498 expref: 21 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339437:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.339439:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.339442:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc3cfa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0498 expref: 21 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.339453:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905000, rc 0) 00000100:00100000:3.0:1787784584.339529:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16556:x1874627657922688:12345-192.168.203.43@tcp:101:cp.0 Request processed in 633us (947us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784584.339537:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 511 00000100:00100000:3.0:1787784584.343035:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657922944 00000100:00100000:3.0:1787784584.343068:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657922944 00000100:00100000:3.0:1787784584.343181:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 512 00000100:00100000:3.0:1787784584.343193:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+20:16556:x1874627657922944:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784584.343222:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.343251:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc3d0f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b049f expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.343441:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc3d16 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.343460:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc3d16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.343474:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc3d16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.343491:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.343499:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.343505:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.354233:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.354251:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.354261:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.354290:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540654a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.354297:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.354307:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.354387:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff989510520400) 00010000:00010000:3.0:1787784584.354391:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.354397:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.354406:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d1d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787784584.360861:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784584.364938:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657923456 00000100:00100000:3.0:1787784584.364950:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657923456 00010000:00010000:3.0:1787784584.365074:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006c1c083c/0xb0ce63137acc3d39 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04a6 expref: 9 pid: 10688 timeout: 475 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784584.365126:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00100000:3.0:1787784584.365138:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657923456:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784584.365191:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784584.365222:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006c1c083c/0xb0ce63137acc3d39 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04a6 expref: 9 pid: 10688 timeout: 475 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.365239:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006c1c083c/0xb0ce63137acc3d39 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04a6 expref: 9 pid: 10688 timeout: 475 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.365296:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784584.365302:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784584.365312:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000caffad29/0xb0ce63137acc3d40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04ad expref: 9 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.365325:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000caffad29/0xb0ce63137acc3d40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04ad expref: 9 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.365380:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000caffad29/0xb0ce63137acc3d40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04ad expref: 9 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784584.365396:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668314880:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784584.365546:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784584.372493:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668314880:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784584.372527:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784584.372588:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2357:x1874627657923456:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 7441us (7737us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784584.372599:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00100000:3.0:1787784584.407193:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657923840 00000100:00100000:3.0:1787784584.407207:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657923840 00000100:00100000:3.0:1787784584.407283:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 513 00000100:00100000:3.0:1787784584.407579:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018370c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+51:16559:x1874627657923840:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784584.407609:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.407640:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d47 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b04b4 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407769:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407786:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407798:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407815:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407821:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407827:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407958:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655600/0xb0ce63137acc3d5c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407967:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655600/0xb0ce63137acc3d5c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.407974:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655600/0xb0ce63137acc3d5c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784584.408030:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xc:0x0]: returning size 1516 00000004:00200000:3.0:1787784584.408035:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784584.408056:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989540655600/0xb0ce63137acc3d5c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408080:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655600/0xb0ce63137acc3d5c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408088:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408095:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408103:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408109:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.408114:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408121:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408135:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:3.0:1787784584.408139:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.408144:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408207:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3d4e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408223:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408228:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408241:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408247:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784584.408249:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408256:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408265:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff989510520400) 00010000:00010000:3.0:1787784584.408267:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784584.408270:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408276:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3d55 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408313:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d47 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04b4 expref: 53 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408328:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655600/0xb0ce63137acc3d5c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04b4 expref: 52 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.408336:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:3.0:1787784584.408338:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784584.408341:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655600, rc 0) 00000100:00100000:3.0:1787784584.408435:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018370c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+52:16559:x1874627657923840:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 861us (1602us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784584.411561:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 513 00000100:00100000:3.0:1787784584.412436:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657923968 00000100:00100000:3.0:1787784584.412448:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657923968 00000100:00100000:3.0:1787784584.412517:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 133 00000100:00100000:3.0:1787784584.412529:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657923968:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784584.412552:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784584.412577:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3d63 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b04bb expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784584.412656:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668315008:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784584.412774:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef300 going to sleep for 11 seconds 00002000:00010000:3.0:1787784584.415633:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size: 0 -> 1516 00002000:00010000:3.0:1787784584.415644:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb mtime: 0 -> 1787784583 00002000:00010000:3.0:1787784584.415647:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb atime: 0 -> 1787784583 00002000:00010000:3.0:1787784584.415650:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb ctime: 0 -> 1787784583 00000100:00100000:3.0:1787784584.415689:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668315008:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784584.415741:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3d63 lrc: 1/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04bb expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.415771:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3d63 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04bb expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784584.415787:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655800, rc 0) 00000100:00100000:3.0:1787784584.416180:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657923968:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3651us (3864us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784584.416215:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 133 00000100:00100000:3.0:1787784584.699695:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989521759680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668315392:0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.701185:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784584.701257:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530505680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668315904:0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.701294:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784584.701333:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784584.701341:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668315776:0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.702173:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784584.702227:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989521759680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668315392:0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.702873:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668315904 00000100:00100000:3.0:1787784584.702882:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668315904 00000100:00100000:3.0:1787784584.702939:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668315520 00000100:00100000:3.0:1787784584.702942:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668315520 00000100:00100000:3.0:1787784584.702965:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1787784584.702973:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668315904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.703053:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668315904:12345-0@lo:400:kworker.0 Request processed in 80us (1758us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784584.703060:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1787784584.703077:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1787784584.703083:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668315520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.703119:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668315520:12345-0@lo:400:kworker.0 Request processed in 37us (782us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784584.703125:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00080000:3.0:1787784584.703180:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784584.703213:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530505680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668315904:0@lo:400:kworker.0 00000100:00080000:3.0:1787784584.705177:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784584.705230:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668315776:0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.711517:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668315648 00000100:00100000:3.0:1787784584.711539:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668315648 00000100:00100000:3.0:1787784584.713336:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:3.0:1787784584.713354:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668315648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784584.713500:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668315648:12345-0@lo:400:kworker.0 Request processed in 151us (6282us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784584.713508:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:3.0:1787784585.405365:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657924608 00000100:00100000:3.0:1787784585.405375:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657924608 00000100:00100000:3.0:1787784585.405433:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:3.0:1787784585.405444:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+23:16569:x1874627657924608:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784585.405489:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xc:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784585.405632:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+23:16569:x1874627657924608:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 188us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.405641:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:3.0:1787784585.424800:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657924864 00000100:00100000:3.0:1787784585.424820:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657924864 00000100:00100000:3.0:1787784585.425129:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:3.0:1787784585.425174:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+23:16569:x1874627657924864:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784585.425203:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784585.425230:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3d7f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b04d0 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.425364:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d86 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.425395:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d86 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.425408:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d86 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784585.425423:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xc:0x0]: returning size 1516 00010000:00010000:3.0:1787784585.425593:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3d7f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04d0 expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.429000:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d86 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04d0 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.429017:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:3.0:1787784585.429020:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784585.429024:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540654a00, rc 0) 00000100:00100000:3.0:1787784585.429133:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:16569:x1874627657924864:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3984us (6920us total) trans 4294967677 rc 0/0 00000100:00100000:3.0:1787784585.429162:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:3.0:1787784585.463752:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657924992 00000100:00100000:3.0:1787784585.463770:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657924992 00000100:00100000:3.0:1787784585.464068:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 13 00000100:00100000:3.0:1787784585.464080:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2357:x1874627657924992:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784585.464100:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784585.464133:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654a00/0xb0ce63137acc3d86 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04d0 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.464188:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:3.0:1787784585.464194:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784585.464272:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2357:x1874627657924992:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 195us (15239us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.464280:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 13 00000100:00100000:3.0:1787784585.468466:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895407383c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668316160:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784585.468494:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784585.533167:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657925376 00000100:00100000:3.0:1787784585.533182:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657925376 00010000:00010000:3.0:1787784585.533238:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000caffad29/0xb0ce63137acc3d40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04ad expref: 9 pid: 6669 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784585.533300:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 166 00000100:00100000:3.0:1787784585.533313:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657925376:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784585.533334:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784585.533365:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000caffad29/0xb0ce63137acc3d40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04ad expref: 9 pid: 6669 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533383:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000caffad29/0xb0ce63137acc3d40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04ad expref: 9 pid: 6669 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533415:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000caffad29/0xb0ce63137acc3d40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04ad expref: 9 pid: 6669 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533441:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784585.533447:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784585.533454:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533466:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533501:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784585.533871:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627657925376:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 558us (1415us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.533882:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 166 00010000:00010000:3.0:1787784585.533917:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533927:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533957:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533967:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.533976:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.536524:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784585.536532:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.536544:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.536570:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.536593:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047d5b2a1/0xb0ce63137acc3d94 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784585.550178:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657925632 00000100:00100000:3.0:1787784585.550192:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657925632 00000100:00100000:3.0:1787784585.550260:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:3.0:1787784585.550272:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+24:16569:x1874627657925632:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784585.550423:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+24:16569:x1874627657925632:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 152us (411us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.550432:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:3.0:1787784585.553560:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657925760 00000100:00100000:3.0:1787784585.553581:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657925760 00000100:00100000:3.0:1787784585.553645:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 136 00000100:00100000:3.0:1787784585.553655:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2357:x1874627657925760:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784585.553703:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784585.553728:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003851d49a/0xb0ce63137acc3d9b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b04d7 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784585.553759:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 1516, mtime: 0x6a8f6d87, atime: 0x0, ctime: 0x6a8f6d87, blocks: 0x1 00010000:00010000:3.0:1787784585.553783:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=376s v=(0 0) 00010000:00010000:3.0:1787784585.553788:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784585.553799:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003851d49a/0xb0ce63137acc3d9b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04d7 expref: 8 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.553818:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003851d49a/0xb0ce63137acc3d9b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04d7 expref: 8 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.553833:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540654a00, rc 0) 00000100:00100000:3.0:1787784585.553890:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627657925760:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 236us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.553897:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 136 00000100:00100000:3.0:1787784585.557182:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657926016 00000100:00100000:3.0:1787784585.557193:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657926016 00000100:00100000:3.0:1787784585.557258:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00100000:3.0:1787784585.557270:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+24:16569:x1874627657926016:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784585.557402:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+24:16569:x1874627657926016:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 134us (384us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.557411:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00100000:3.0:1787784585.560243:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657926144 00000100:00100000:3.0:1787784585.560261:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657926144 00002000:00100000:3.0:1787784585.560322:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xb:0x0] (0->4095) req@ffff9895391292c0 x1874627657926144/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:466/0 lens 488/0 e 0 to 0 dl 1787784596 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784585.560333:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895391292c0 with x1874627657926144 ext(0->4095) 00002000:00010000:3.0:1787784585.560339:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9895391292c0 00000100:00100000:3.0:1787784585.560381:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 50 00000100:00100000:3.0:1787784585.560392:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391292c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657926144:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784585.560664:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391292c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657926144:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 273us (563us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784585.560672:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 50 00002000:00100000:3.0:1787784585.560679:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xb:0x0] (0->4095) req@ffff9895391292c0 x1874627657926144/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:466/0 lens 488/4536 e 0 to 0 dl 1787784596 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784585.560687:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895391292c0 with x1874627657926144 ext(0->4095) 00002000:00010000:3.0:1787784585.560691:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9895391292c0 00000100:00100000:3.0:1787784585.587361:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784585.608611:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657926400 00000100:00100000:3.0:1787784585.608629:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657926400 00000100:00100000:3.0:1787784585.612289:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657926528 00000100:00100000:3.0:1787784585.612302:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657926528 00000100:00100000:3.0:1787784585.612364:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 137 00000100:00100000:3.0:1787784585.612377:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657926528:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784585.612436:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612458:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784585.612483:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784585.612491:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612509:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612552:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612562:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612573:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612582:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784585.612587:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612598:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612616:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.612625:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000919e7555/0xb0ce63137acc3db0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784585.612827:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657926528:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 451us (850us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.612836:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 137 00000100:00100000:3.0:1787784585.643114:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657926912 00000100:00100000:3.0:1787784585.643135:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657926912 00010000:00010000:3.0:1787784585.643211:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3da9 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04de expref: 26 pid: 5811 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784585.643255:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 167 00000100:00100000:3.0:1787784585.643266:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2357:x1874627657926912:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784585.643286:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784585.643294:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3da9 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04de expref: 26 pid: 5811 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.643309:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3da9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04de expref: 26 pid: 5811 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.643338:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:3.0:1787784585.643344:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.643355:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.643363:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.643398:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784585.643473:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2357:x1874627657926912:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 209us (2157us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784585.643481:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 167 00010000:00010000:3.0:1787784585.643707:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3da9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04de expref: 25 pid: 5811 timeout: 476 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784585.643723:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ad2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668316288:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784585.643748:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.643760:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.643768:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.644037:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.644044:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.644050:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.644056:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784585.644061:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.644081:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.644091:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:3.0:1787784585.644094:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784585.644098:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784585.644104:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3db7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xc:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784585.644287:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:16569:x1874627657926784:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 18166us (18344us total) trans 4294967678 rc 0/0 00000100:00100000:3.0:1787784585.644297:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 522 00000100:00100000:3.0:1787784585.651800:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784585.651843:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657927168 00000100:00100000:3.0:1787784585.651855:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657927168 00000100:00100000:3.0:1787784585.651911:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 116 00000100:00100000:3.0:1787784585.651922:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:16569:x1874627657927168:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784585.652266:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:16569:x1874627657927168:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 345us (473us total) trans 4294967679 rc 0/0 00000100:00100000:3.0:1787784585.652276:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 116 00000001:00080000:3.0:1787784586.332302:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967676 is committed 00000001:00080000:3.0:1787784586.332320:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967674 is committed 00000001:00080000:3.0:1787784586.332332:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967679 is committed 00000100:00100000:3.0:1787784587.761485:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784587.761516:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391283c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668316416:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784587.761625:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1787784587.762502:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784587.943772:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657927424 00000100:00100000:3.0:1787784587.943785:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657927424 00000100:00100000:3.0:1787784587.943883:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:3.0:1787784587.943898:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+52:16580:x1874627657927424:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784587.943927:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784587.943955:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3dbe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b04e5 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784587.944105:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3dc5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784587.944123:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3dc5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784587.944139:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3dc5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784587.944178:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xc:0x0]: returning size 1516 00010000:00010000:3.0:1787784587.944493:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3dbe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04e5 expref: 55 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784587.944514:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3dc5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04e5 expref: 54 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784587.944524:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:3.0:1787784587.944527:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784587.944532:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655e00, rc 0) 00000100:00100000:3.0:1787784587.944636:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+54:16580:x1874627657927424:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 742us (1022us total) trans 4294967680 rc 0/0 00000100:00100000:3.0:1787784587.944645:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:3.0:1787784587.958124:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784587.967183:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657927680 00000100:00100000:3.0:1787784587.967197:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657927680 00000100:00100000:3.0:1787784587.967282:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 14 00000100:00100000:3.0:1787784587.967294:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+54:2355:x1874627657927680:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784587.967316:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784587.967349:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3dc5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04e5 expref: 54 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784587.967371:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:3.0:1787784587.968265:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784587.968290:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+54:2355:x1874627657927680:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1000us (1397us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784587.968298:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 14 00000100:00100000:3.0:1787784588.051060:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657928064 00000100:00100000:3.0:1787784588.051074:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657928064 00000100:00100000:3.0:1787784588.051180:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 319 00000100:00100000:3.0:1787784588.051193:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657928064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784588.051284:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657928064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 94us (733us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784588.051292:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 319 00000040:00080000:3.0:1787784590.770583:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784590.770596:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784590.771322:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 12 count 2 00000040:00080000:3.0:1787784590.771337:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:3.0:1787784590.921419:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967323 is committed 00010000:00010000:3.0:1787784591.284525:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784592.883283:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668317312 00000100:00100000:3.0:1787784592.883295:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668317312 00000100:00100000:3.0:1787784592.883383:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:3.0:1787784592.883396:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668317312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784592.883458:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784592.891353:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668317312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 7944us (8292us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784592.891373:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:3.0:1787784593.160663:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657928832 00000100:00100000:3.0:1787784593.160826:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657928832 00000100:00100000:3.0:1787784593.160883:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 321 00000100:00100000:3.0:1787784593.160893:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657928832:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784593.160997:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657928832:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 104us (424us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784593.161005:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 321 00010000:00010000:3.0:1787784593.334080:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784594.931259:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668317696:0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.931298:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784594.931328:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784594.931333:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668317824:0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.931357:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784594.931388:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668317696 00000100:00100000:3.0:1787784594.931392:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668317696 00000100:00100000:3.0:1787784594.931430:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668317824 00000100:00100000:3.0:1787784594.931433:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668317824 00000100:00100000:3.0:1787784594.931829:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668318080 00000100:00100000:3.0:1787784594.931833:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668318080 00000100:00100000:3.0:1787784594.931850:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:3.0:1787784594.931856:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668317696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.931900:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668317696:12345-0@lo:400:kworker.0 Request processed in 44us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784594.931906:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:3.0:1787784594.931920:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:3.0:1787784594.931937:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668317824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.931963:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668317824:12345-0@lo:400:kworker.0 Request processed in 28us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784594.931968:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:3.0:1787784594.931978:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:3.0:1787784594.931983:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668318080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.932014:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668318080:12345-0@lo:400:kworker.0 Request processed in 30us (440us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784594.932019:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:3.0:1787784594.932053:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668317824:0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.932065:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668317696:0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.932092:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668318080:0@lo:400:kworker.0 00000100:00100000:3.0:1787784594.932506:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668317952:0@lo:400:kworker.0 00000100:00100000:3.0:1787784596.031064:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784596.069486:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784596.069582:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000040:00080000:3.0:1787784596.913842:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784596.913856:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784596.914587:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:3.0:1787784596.914613:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784598.001494:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668318208:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784598.001599:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784598.283317:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784598.283387:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784598.283437:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784598.283472:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784598.283517:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784598.283560:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784598.283592:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784598.283624:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657931392 00000100:00100000:3.0:1787784598.283629:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657931392 00000100:00100000:3.0:1787784598.283942:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657931008 00000100:00100000:3.0:1787784598.283947:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657931008 00000100:00080000:3.0:1787784598.292283:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784598.292292:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784598.292295:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784603.121823:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668319104 00000100:00100000:3.0:1787784603.121831:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668319104 00000100:00100000:3.0:1787784603.121917:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1787784603.121927:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668319104:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784603.121965:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784603.125076:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668319104:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3143us (3436us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784603.125093:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1787784603.403320:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784603.403370:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784603.403409:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784603.403448:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784603.403467:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784603.403490:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784603.403532:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784603.403560:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657932032 00000100:00100000:3.0:1787784603.403569:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657932032 00000100:00100000:3.0:1787784603.403618:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657931648 00000100:00100000:3.0:1787784603.403621:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657931648 00000100:00100000:3.0:1787784603.403729:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 219 00000100:00100000:3.0:1787784603.403740:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627657932032:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784603.403814:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627657932032:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 76us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784603.403822:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 219 00000100:00100000:3.0:1787784603.403839:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 220 00000100:00100000:3.0:1787784603.403847:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+52:2357:x1874627657931648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784603.403886:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+52:2357:x1874627657931648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784603.403894:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 220 00000100:00100000:3.0:1787784605.169404:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402186580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668319232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784605.169473:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1787784605.169857:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784605.169896:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402186580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668319232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784605.170144:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784605.175002:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668319360 00000100:00100000:3.0:1787784605.175011:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668319360 00000001:02000400:3.0:1787784608.045803:0:15351:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg343-client.virtnet: executing get_client_uuid /mnt/lustre2 00000100:00100000:3.0:1787784608.530308:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784608.534199:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784608.535251:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784608.535393:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784608.535415:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784608.535444:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784608.535469:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784608.535495:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657932928 00000100:00100000:3.0:1787784608.535513:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657932928 00000100:00100000:1.0:1787784608.536261:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 161 00000100:00100000:1.0:1787784608.536286:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657932672:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784608.536367:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657932672:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 82us (951us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784608.536376:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 161 00000100:00100000:1.0:1787784608.536393:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 162 00000100:00100000:1.0:1787784608.536400:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2357:x1874627657933184:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784608.536425:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2357:x1874627657933184:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (955us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784608.536431:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 162 00000100:00100000:1.0:1787784608.536812:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657932416 00000100:00100000:1.0:1787784608.536818:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657932416 00000100:00100000:1.0:1787784608.536873:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 327 00000100:00100000:1.0:1787784608.536881:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627657932416:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784608.537071:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627657932416:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 190us (1677us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784608.537079:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 327 00000100:00100000:1.0:1787784608.538442:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 163 00000100:00100000:1.0:1787784608.538460:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2357:x1874627657933056:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784608.538875:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2357:x1874627657933056:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 419us (3625us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784608.538883:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 163 00000100:00100000:3.0:1787784608.540071:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 225 00000100:00100000:3.0:1787784608.540088:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627657932928:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784608.540185:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627657932928:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 103us (4739us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784608.540193:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 225 00010000:00010000:3.0:1787784608.690346:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784609.243264:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657933824 00000100:00100000:1.0:1787784609.243278:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657933824 00000100:00100000:1.0:1787784609.243357:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 530 00000100:00100000:1.0:1787784609.243389:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+52:17701:x1874627657933824:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784609.243423:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784609.243453:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc3e20 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b04fa expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.243540:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc3e27 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.243557:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc3e27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.243570:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc3e27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784609.243651:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 7680 00000004:00200000:1.0:1787784609.243656:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 13 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784609.243685:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc3e27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.243716:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc3e20 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04fa expref: 54 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.243737:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc3e27 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04fa expref: 53 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.243746:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649300) 00010000:00010000:1.0:1787784609.243749:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784609.243753:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010aca00, rc 0) 00000100:00100000:1.0:1787784609.243847:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+53:17701:x1874627657933824:12345-192.168.203.43@tcp:101:bash.0 Request processed in 460us (1171us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784609.243866:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 530 00000100:00100000:3.0:1787784609.268347:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784609.291972:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657934208 00000100:00100000:3.0:1787784609.291984:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657934208 00000100:00100000:3.0:1787784609.297995:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784609.298194:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 534 00000100:00100000:3.0:1787784609.298209:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+20:17702:x1874627657934336:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784609.298233:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784609.298256:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3e6d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0516 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298342:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298357:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298368:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298380:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298386:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298392:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298518:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654400/0xb0ce63137acc3e82 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298526:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540654400/0xb0ce63137acc3e82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298533:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654400/0xb0ce63137acc3e82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784609.298573:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x21:0x0]: returning size 512 00000004:00200000:3.0:1787784609.298577:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784609.298727:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989540654400/0xb0ce63137acc3e82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298752:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298758:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298764:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298770:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784609.298774:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298782:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298796:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649300) 00010000:00010000:3.0:1787784609.298799:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784609.298804:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298810:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3e74 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298956:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298963:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298968:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298980:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784609.298983:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298988:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.298996:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4f27b7ce (ffff989510520400) 00010000:00010000:3.0:1787784609.298998:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784609.299000:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.299005:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3e7b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.299041:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3e6d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0516 expref: 22 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.299055:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654400/0xb0ce63137acc3e82 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0516 expref: 21 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.299062:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff98950a162300) 00010000:00010000:3.0:1787784609.299064:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784609.299067:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540654400, rc 0) 00000100:00100000:3.0:1787784609.299207:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+21:17702:x1874627657934336:12345-192.168.203.43@tcp:101:rm.0 Request processed in 997us (1219us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784609.299215:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 534 00000100:00100000:3.0:1787784609.301782:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784609.354745:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657935104 00000100:00100000:3.0:1787784609.354759:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657935104 00000100:00100000:3.0:1787784609.354851:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 539 00000100:00100000:3.0:1787784609.354863:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+21:17703:x1874627657935104:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784609.354886:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784609.354907:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655000/0xb0ce63137acc3edd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0539 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.355035:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.355051:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.355062:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.355081:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.355088:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.355095:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784609.355847:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xd:0x0], index 1: 0 00010000:00010000:3.0:1787784609.356019:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3ef2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356029:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3ef2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356045:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3ef2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356079:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356088:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356095:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356102:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784609.356106:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356114:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356126:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff98950a162300) 00010000:00010000:3.0:1787784609.356129:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784609.356134:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356140:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3ee4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784609.356179:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eadfddec reply state 00000000fcabc416 transno 4294967685 ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784609.356188:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787784609.356223:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655000/0xb0ce63137acc3edd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0539 expref: 24 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356236:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3ef2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0539 expref: 23 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.356244:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:3.0:1787784609.356247:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784609.356249:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655e00, rc 0) 00000100:00100000:3.0:1787784609.356324:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+24:17703:x1874627657935104:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1464us (4605us total) trans 4294967685 rc 0/0 00000100:00100000:3.0:1787784609.356332:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 539 00000100:00100000:1.0:1787784609.362229:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784609.362264:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657935232 00000100:00100000:1.0:1787784609.362273:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657935232 00000100:00100000:1.0:1787784609.362326:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 138 00000100:00100000:1.0:1787784609.362333:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2355:x1874627657935232:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784609.362347:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784609.362363:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0540 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784609.362384:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784609.362399:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=400s v=(0 0) 00010000:00010000:1.0:1787784609.362403:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784609.362412:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 7 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.362431:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 7 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.362445:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac600, rc 0) 00000100:00100000:1.0:1787784609.362497:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2355:x1874627657935232:12345-192.168.203.43@tcp:101:cp.0 Request processed in 164us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784609.362502:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 138 00010000:00010000:3.0:1787784609.362984:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.362994:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.363001:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.363007:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784609.363012:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.363018:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.363030:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x3953761c (ffff989510520400) 00010000:00010000:3.0:1787784609.363034:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784609.363051:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 2/0,0 mode: --/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.363057:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3eeb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784609.366933:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784609.367060:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657935360 00000100:00100000:1.0:1787784609.367069:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657935360 00000100:00100000:1.0:1787784609.367121:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 139 00000100:00100000:1.0:1787784609.367180:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+7:17703:x1874627657935360:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784609.367200:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784609.367218:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0547 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.367248:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 8 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.367256:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784609.367260:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784609.367266:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 4/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 8 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.367299:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 8 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.367313:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 8 pid: 13825 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784609.367327:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668320256:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784609.367404:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066db40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784609.370184:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668320256:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784609.370213:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 8 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784609.370254:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac400, rc 0) 00000100:00100000:1.0:1787784609.370307:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:17703:x1874627657935360:12345-192.168.203.43@tcp:101:cp.0 Request processed in 3164us (3383us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784609.370316:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 139 00000100:00100000:3.0:1787784609.370481:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657935488 00000100:00100000:3.0:1787784609.370488:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657935488 00010000:00010000:3.0:1787784609.370516:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 8 pid: 13825 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784609.370541:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 169 00000100:00100000:3.0:1787784609.370550:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657935488:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784609.370563:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784609.370583:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 8 pid: 13825 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.370592:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 8 pid: 13825 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.370612:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008259c19d/0xb0ce63137acc3f00 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0540 expref: 8 pid: 13825 timeout: 500 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.370634:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784609.370639:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784609.370648:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 7 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.370680:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 7 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.370711:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 7 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784609.371097:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ecb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668320384:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784609.371187:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd780 going to sleep for 11 seconds 00000100:00100000:3.0:1787784609.376564:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ecb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668320384:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784609.376590:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784609.376658:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657935488:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 6109us (6608us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784609.376669:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 169 00000100:00100000:3.0:1787784609.420788:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784609.420903:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657936000 00000100:00100000:1.0:1787784609.420913:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657936000 00000100:00100000:1.0:1787784609.420984:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 140 00000100:00100000:1.0:1787784609.420995:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527ded050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2357:x1874627657936000:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784609.421013:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784609.421037:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e597a957/0xb0ce63137acc3f2a lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0555 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784609.421095:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668320512:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784609.421240:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066db40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784609.431648:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657936128 00000100:00100000:3.0:1787784609.431659:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657936128 00000100:00100000:3.0:1787784609.431780:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 541 00000100:00100000:3.0:1787784609.431792:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+55:17706:x1874627657936128:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784609.432031:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784609.432063:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3f31 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b055c expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.432171:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655000/0xb0ce63137acc3f38 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.432196:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655000/0xb0ce63137acc3f38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.432208:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655000/0xb0ce63137acc3f38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784609.432266:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x21:0x0]: returning size 512 00000004:00200000:3.0:1787784609.432270:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784609.432290:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989540655000/0xb0ce63137acc3f38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.432316:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3f31 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b055c expref: 57 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.432334:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655000/0xb0ce63137acc3f38 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b055c expref: 56 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784609.432342:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff98950a162300) 00010000:00010000:3.0:1787784609.432345:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784609.432349:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655000, rc 0) 00000100:00100000:3.0:1787784609.432406:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:17706:x1874627657936128:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 617us (820us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784609.432413:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 541 00000100:00100000:3.0:1787784609.437767:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784609.444213:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657936256 00000100:00100000:1.0:1787784609.444228:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657936256 00000100:00100000:1.0:1787784609.444301:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 141 00000100:00100000:1.0:1787784609.444312:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2356:x1874627657936256:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784609.444330:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784609.444348:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fed6a013/0xb0ce63137acc3f3f lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0563 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784609.444406:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a91e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668320640:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784609.444504:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784609.460405:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784609.713714:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784610.243465:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:1.0:1787784610.243989:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784610.253292:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657936896 00000100:00100000:3.0:1787784610.253302:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657936896 00000100:00100000:3.0:1787784610.253379:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 544 00000100:00100000:3.0:1787784610.253390:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:17716:x1874627657936896:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784610.253411:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784610.253432:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3f46 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b056a expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.253542:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f4d lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.253568:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f4d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.253581:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f4d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784610.253594:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xd:0x0]: returning size 1516 00010000:00010000:3.0:1787784610.253722:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3f46 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b056a expref: 27 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.253751:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f4d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b056a expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.254490:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:3.0:1787784610.254494:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784610.254498:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655800, rc 0) 00000100:00100000:3.0:1787784610.254664:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+26:17716:x1874627657936896:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1275us (2206us total) trans 4294967687 rc 0/0 00000100:00100000:3.0:1787784610.254674:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 544 00000100:00100000:3.0:1787784610.260303:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784610.260362:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784610.260397:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657937152 00000100:00100000:3.0:1787784610.260402:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657937152 00000100:00100000:3.0:1787784610.260451:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 16 00000100:00100000:3.0:1787784610.260464:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2356:x1874627657937152:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784610.260481:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784610.260504:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f4d lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b056a expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.260530:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:3.0:1787784610.260536:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784610.260584:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2356:x1874627657937152:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 124us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784610.260593:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 16 00000100:00100000:3.0:1787784610.269124:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784610.269341:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657937280 00000100:00100000:3.0:1787784610.269351:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657937280 00002000:00100000:3.0:1787784610.269401:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xd:0x0] (0->1515) req@ffff9894030ed680 x1874627657937280/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:491/0 lens 2008/0 e 0 to 0 dl 1787784621 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784610.269413:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894030ed680 with x1874627657937280 ext(0->1515) 00010000:00010000:3.0:1787784610.269423:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 8 pid: 10688 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.269444:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 8 pid: 10688 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784610.269452:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9894030ed680 00000100:00100000:3.0:1787784610.269501:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 2 00000100:00100000:3.0:1787784610.269513:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ed680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657937280:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:3.0:1787784610.271650:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ed680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657937280:12345-192.168.203.43@tcp:4:cp.0 Request processed in 2133us (2530us total) trans 4294967328 rc 0/0 00000100:00100000:3.0:1787784610.271660:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 2 00002000:00100000:3.0:1787784610.271669:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xd:0x0] (0->1515) req@ffff9894030ed680 x1874627657937280/t4294967328(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:491/0 lens 2008/448 e 0 to 0 dl 1787784621 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784610.271676:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894030ed680 with x1874627657937280 ext(0->1515) 00010000:00010000:3.0:1787784610.271684:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 10 pid: 10688 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.271709:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d7e505dd/0xb0ce63137acc3f07 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0547 expref: 10 pid: 10688 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784610.271715:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9894030ed680 00000100:00100000:3.0:1787784610.275873:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784610.279070:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784610.279140:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657937664 00000100:00100000:1.0:1787784610.279176:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657937664 00000100:00100000:1.0:1787784610.279226:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 545 00000100:00100000:1.0:1787784610.279235:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:17716:x1874627657937664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784610.279540:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:17716:x1874627657937664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 305us (478us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784610.279549:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 545 00000100:00100000:1.0:1787784610.283299:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657937792 00000100:00100000:1.0:1787784610.283309:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657937792 00000100:00100000:1.0:1787784610.283367:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 143 00000100:00100000:1.0:1787784610.283376:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657937792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784610.283394:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784610.283415:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000335e89d/0xb0ce63137acc3f62 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0571 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784610.283441:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 1516, mtime: 0x6a8f6da0, atime: 0x0, ctime: 0x6a8f6da0, blocks: 0x1 00010000:00010000:1.0:1787784610.283461:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=401s v=(0 0) 00010000:00010000:1.0:1787784610.283467:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784610.283477:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000335e89d/0xb0ce63137acc3f62 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0571 expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.283493:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000335e89d/0xb0ce63137acc3f62 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0571 expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.283507:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad600, rc 0) 00000100:00100000:1.0:1787784610.283575:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657937792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 200us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784610.283583:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 143 00000100:00100000:3.0:1787784610.287392:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657938048 00000100:00100000:3.0:1787784610.287401:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657938048 00000100:00100000:3.0:1787784610.287459:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 546 00000100:00100000:3.0:1787784610.287471:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+25:17716:x1874627657938048:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784610.287573:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+25:17716:x1874627657938048:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 104us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784610.287581:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 546 00000100:00100000:3.0:1787784610.289510:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668321024 00000100:00100000:3.0:1787784610.289528:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668321024 00000100:00100000:3.0:1787784610.289586:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:3.0:1787784610.289593:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668321024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784610.289610:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 1703936 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784610.289656:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668321024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (326us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784610.289664:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00080000:1.0:1787784610.289952:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784610.292470:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668321152 00000100:00100000:3.0:1787784610.292481:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668321152 00000100:00100000:3.0:1787784610.292527:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:3.0:1787784610.292536:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668321152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784610.299211:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668321280:0@lo:400:kworker.0 00000100:00100000:1.0:1787784610.299287:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784610.299343:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668321280 00000100:00100000:1.0:1787784610.299350:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668321280 00000100:00100000:1.0:1787784610.299396:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:1.0:1787784610.299406:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668321280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784610.299459:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668321280:12345-0@lo:400:kworker.0 Request processed in 54us (172us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784610.299467:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:1.0:1787784610.299524:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ee580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668321024:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784610.299553:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668321280:0@lo:400:kworker.0 00000100:00100000:3.0:1787784610.300193:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668321152:12345-0@lo:400:kworker.0 Request processed in 7655us (9625us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784610.300204:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1787784610.301098:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784610.301234:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657938176 00000100:00100000:1.0:1787784610.301241:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657938176 00002000:00100000:1.0:1787784610.301279:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (0->4095) req@ffff989402a91680 x1874627657938176/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:491/0 lens 488/0 e 0 to 0 dl 1787784621 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784610.301290:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402a91680 with x1874627657938176 ext(0->4095) 00002000:00010000:1.0:1787784610.301296:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989402a91680 00000100:00100000:1.0:1787784610.301319:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 51 00000100:00100000:1.0:1787784610.301327:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657938176:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1787784610.301492:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657938176:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 165us (396us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787784610.301508:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 51 00002000:00100000:1.0:1787784610.301515:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (0->4095) req@ffff989402a91680 x1874627657938176/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:491/0 lens 488/4536 e 0 to 0 dl 1787784621 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784610.301523:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402a91680 with x1874627657938176 ext(0->4095) 00002000:00010000:1.0:1787784610.301526:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989402a91680 00000100:00100000:2.0:1787784610.304988:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668321664:0@lo:400:kworker.0 00000100:00100000:2.0:1787784610.305064:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784610.305120:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668321664 00000100:00100000:2.0:1787784610.305126:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668321664 00000100:00100000:2.0:1787784610.305194:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:2.0:1787784610.305202:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668321664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784610.305253:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668321664:12345-0@lo:400:kworker.0 Request processed in 52us (190us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784610.305260:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00080000:2.0:1787784610.305278:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784610.305306:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668321664:0@lo:400:kworker.0 00000100:00100000:3.0:1787784610.306327:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668321536:0@lo:400:kworker.0 00000100:00100000:3.0:1787784610.306374:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784610.306408:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784610.306414:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668321408:0@lo:400:kworker.0 00000100:00100000:3.0:1787784610.306437:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784610.306539:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668321408 00000100:00100000:1.0:1787784610.306544:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668321408 00000100:00100000:1.0:1787784610.306593:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:1.0:1787784610.306602:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668321408:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784610.306648:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668321408:12345-0@lo:400:kworker.0 Request processed in 47us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784610.306656:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00080000:3.0:1787784610.306710:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784610.306735:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668321408:0@lo:400:kworker.0 00000100:00100000:3.0:1787784610.307340:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668321536 00000100:00100000:3.0:1787784610.307347:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668321536 00000100:00100000:3.0:1787784610.307396:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1787784610.307404:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668321536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784610.307449:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668321536:12345-0@lo:400:kworker.0 Request processed in 46us (1074us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784610.307463:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00080000:3.0:1787784610.307479:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784610.307500:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668321536:0@lo:400:kworker.0 00000100:00100000:3.0:1787784610.310116:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657938304 00000100:00100000:3.0:1787784610.310126:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657938304 00000100:00100000:3.0:1787784610.310199:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 547 00000100:00100000:3.0:1787784610.310207:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:17716:x1874627657938304:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784610.310225:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784610.310243:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc3f69 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0578 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.310271:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f70 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.310282:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f70 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.310291:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f70 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.310365:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc3f69 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0578 expref: 27 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.310377:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f70 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0578 expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.310383:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:3.0:1787784610.310386:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784610.310392:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655800, rc 0) 00000100:00100000:3.0:1787784610.310446:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+26:17716:x1874627657938304:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 240us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784610.310451:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 547 00000100:00100000:1.0:1787784610.313527:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784610.313562:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657938432 00000100:00100000:1.0:1787784610.313572:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657938432 00000100:00100000:1.0:1787784610.313618:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 548 00000100:00100000:1.0:1787784610.313627:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:17716:x1874627657938432:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784610.313659:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xd:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784610.313754:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:17716:x1874627657938432:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 128us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784610.313762:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 548 00000040:00080000:1.0:1787784610.314738:0:15452:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784610.314745:0:15452:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784610.314798:0:15452:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 13 count 2 00000040:00080000:1.0:1787784610.314803:0:15452:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784610.319597:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657938560 00000100:00100000:1.0:1787784610.319607:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657938560 00000100:00100000:1.0:1787784610.319657:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 144 00000100:00100000:1.0:1787784610.319668:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657938560:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784610.319711:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319728:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784610.319733:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784610.319739:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319753:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319783:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319792:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319800:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319807:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ade00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784610.319811:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319844:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.319861:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.320030:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc3f77 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784610.320113:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657938560:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 445us (598us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784610.320122:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 144 00000100:00100000:1.0:1787784610.325091:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657938816 00000100:00100000:1.0:1787784610.325102:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657938816 00000100:00100000:1.0:1787784610.325201:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:1.0:1787784610.325213:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+26:17716:x1874627657938816:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784610.325292:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.325313:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f70 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0578 expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.325323:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 4/0,1 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.325360:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f70 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0578 expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784610.325383:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc3f70 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0578 expref: 26 pid: 5812 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784610.325400:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a92d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668321792:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784610.325480:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d240 going to sleep for 11 seconds 00000100:00100000:3.0:1787784610.340014:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a92d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668321792:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784610.340014:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:3.0:1787784610.340043:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341491:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341502:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341715:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341724:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341731:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341739:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784610.341743:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341751:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341761:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:3.0:1787784610.341772:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784610.341776:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 2/0,0 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784610.341783:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc3f7e lrc: 0/0,0 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784610.342110:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+26:17716:x1874627657938816:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 16898us (17078us total) trans 4294967688 rc 0/0 00000100:00100000:3.0:1787784610.342329:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:1.0:1787784610.345353:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784610.345605:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657939200 00000100:00100000:3.0:1787784610.345617:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657939200 00000100:00100000:3.0:1787784610.345678:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 120 00000100:00100000:3.0:1787784610.345690:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:17716:x1874627657939200:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784610.345922:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:17716:x1874627657939200:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 233us (574us total) trans 4294967689 rc 0/0 00000100:00100000:3.0:1787784610.345931:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 120 00010000:00010000:3.0:1787784610.737405:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784611.761355:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787784611.880275:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967686 is committed 00000001:00080000:2.0:1787784611.880290:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967684 is committed 00000001:00080000:2.0:1787784611.880295:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967689 is committed 00000040:00080000:2.0:1787784612.150297:0:15506:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784612.150306:0:15506:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784612.150360:0:15506:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 13 count 2 00000040:00080000:2.0:1787784612.150366:0:15506:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784612.233036:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784612.235676:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657939456 00000100:00100000:2.0:1787784612.235687:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657939456 00000100:00100000:2.0:1787784612.235763:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 550 00000100:00100000:2.0:1787784612.235774:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+55:17727:x1874627657939456:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784612.235793:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784612.235823:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3f85 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b057f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784612.235915:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3f8c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784612.235939:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3f8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784612.235951:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3f8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784612.235962:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xd:0x0]: returning size 1516 00010000:00010000:2.0:1787784612.236117:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3f85 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b057f expref: 58 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784612.236132:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3f8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b057f expref: 57 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784612.236141:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:2.0:1787784612.236167:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784612.236172:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401099800, rc 0) 00000100:00100000:2.0:1787784612.236230:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:17727:x1874627657939456:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 459us (3203us total) trans 4294967690 rc 0/0 00000100:00100000:2.0:1787784612.236239:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 550 00000100:00080000:3.0:1787784612.236400:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784612.240549:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784612.241575:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657939584 00000100:00100000:1.0:1787784612.241585:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657939584 00000100:00100000:1.0:1787784612.241649:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 551 00000100:00100000:1.0:1787784612.241670:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:17727:x1874627657939584:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:1.0:1787784612.241741:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241766:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241778:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241812:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241819:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241826:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241849:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784612.241854:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241862:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241876:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:1.0:1787784612.241878:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784612.241883:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 2/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.241890:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc3f9a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784612.241964:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:17727:x1874627657939584:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 306us (1423us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784612.241973:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 551 00000100:00100000:1.0:1787784612.254482:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784612.254534:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657939712 00000100:00100000:1.0:1787784612.254544:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657939712 00000100:00100000:1.0:1787784612.254627:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 17 00000100:00100000:1.0:1787784612.254638:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:2356:x1874627657939712:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784612.254655:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784612.254691:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3f8c lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b057f expref: 57 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784612.254712:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:1.0:1787784612.254724:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784612.254773:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:2356:x1874627657939712:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 137us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784612.254781:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 17 00000100:00100000:3.0:1787784612.259129:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:3.0:1787784612.787453:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784613.361341:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784613.361377:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668321920:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784613.361443:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784613.361747:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668321920 00000100:00100000:2.0:1787784613.361756:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668321920 00000100:00100000:2.0:1787784613.361838:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1787784613.361848:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668321920:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784613.361903:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784613.361950:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668321920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (506us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784613.361973:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00080000:3.0:1787784613.362028:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784613.362063:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668321920:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784613.639856:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784613.639903:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784613.639931:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784613.639957:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784613.639982:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657940224 00000100:00100000:3.0:1787784613.639989:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657940224 00000100:00100000:3.0:1787784613.640033:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 229 00000100:00100000:3.0:1787784613.640043:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:2356:x1874627657940224:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784613.640119:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:2356:x1874627657940224:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 76us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784613.640126:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 229 00000100:00100000:3.0:1787784613.640203:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657940480 00000100:00100000:3.0:1787784613.640208:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657940480 00000100:00100000:3.0:1787784613.640245:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657940736 00000100:00100000:3.0:1787784613.640249:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657940736 00000100:00100000:3.0:1787784613.640271:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00100000:3.0:1787784613.640277:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2356:x1874627657940480:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784613.640304:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2356:x1874627657940480:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784613.640311:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00100000:3.0:1787784613.640322:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 166 00000100:00100000:3.0:1787784613.640329:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2356:x1874627657940736:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784613.640353:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2356:x1874627657940736:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784613.640359:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 166 00000100:00080000:3.0:1787784613.640492:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784613.640496:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784613.643244:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657940352 00000100:00100000:3.0:1787784613.643254:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657940352 00000100:00100000:3.0:1787784613.643318:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 167 00000100:00100000:3.0:1787784613.643329:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657940352:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784613.643394:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657940352:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 67us (1448us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784613.643401:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 167 00000100:00080000:2.0:1787784613.646865:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784613.653318:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657940608 00000100:00100000:3.0:1787784613.653333:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657940608 00000100:00100000:3.0:1787784613.653491:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:3.0:1787784613.653503:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406670c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627657940608:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784613.653583:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406670c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627657940608:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 83us (4530us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784613.653590:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:3.0:1787784613.653697:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784613.653772:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657940864 00000100:00100000:3.0:1787784613.653776:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657940864 00000100:00100000:3.0:1787784613.653819:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 168 00000100:00100000:3.0:1787784613.653825:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2357:x1874627657940864:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784613.653855:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2357:x1874627657940864:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784613.653861:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 168 00000100:00080000:2.0:1787784613.653925:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784613.653930:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784613.810660:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784615.410082:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a930c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668322048:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784615.410226:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784615.410380:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668322048 00000100:00100000:1.0:1787784615.410390:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668322048 00000100:00100000:1.0:1787784615.410470:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:1.0:1787784615.410482:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668322048:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784615.410530:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784615.410593:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668322048:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784615.410601:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00080000:2.0:1787784615.410674:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784615.410724:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a930c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668322048:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784615.410748:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784615.410755:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668322176:0@lo:400:kworker.0 00000100:00100000:2.0:1787784615.410809:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784615.410843:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668322560:0@lo:400:kworker.0 00000100:00100000:2.0:1787784615.410875:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784615.410931:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668322176 00000100:00100000:2.0:1787784615.410936:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668322176 00000100:00100000:2.0:1787784615.410976:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:2.0:1787784615.410986:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668322176:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784615.411032:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668322176:12345-0@lo:400:kworker.0 Request processed in 49us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784615.411042:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:2.0:1787784615.411092:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668322176:0@lo:400:kworker.0 00000100:00100000:3.0:1787784615.411126:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668322560 00000100:00100000:3.0:1787784615.411138:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668322560 00000100:00080000:2.0:1787784615.411174:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784615.411179:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784615.411205:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:3.0:1787784615.411215:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668322560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784615.411269:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668322304:0@lo:400:kworker.0 00000100:00100000:2.0:1787784615.417918:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668322560:0@lo:400:kworker.0 00000100:00100000:3.0:1787784615.417982:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668322560:12345-0@lo:400:kworker.0 Request processed in 6763us (7101us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784615.417991:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784615.418016:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1787784615.418037:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784615.418052:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668322432:0@lo:400:kworker.0 00000100:00100000:1.0:1787784615.418088:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784615.419184:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668322304 00000100:00100000:2.0:1787784615.419184:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668322432 00000100:00100000:3.0:1787784615.419199:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668322304 00000100:00100000:2.0:1787784615.419212:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668322432 00000100:00100000:3.0:1787784615.419263:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1787784615.419275:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1787784615.419285:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668322432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784615.419338:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668322304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784615.419426:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668322432:12345-0@lo:400:kworker.0 Request processed in 142us (1336us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784615.419433:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00080000:2.0:1787784615.419458:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784615.419467:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668322304:12345-0@lo:400:kworker.0 Request processed in 194us (1477us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784615.419469:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668322432:0@lo:400:kworker.0 00000100:00100000:3.0:1787784615.419484:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:1.0:1787784615.419490:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668322304:0@lo:400:kworker.0 00000100:00100000:3.0:1787784615.420314:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668322688:0@lo:400:kworker.0 00010000:00010000:3.0:1787784615.857302:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784616.730229:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657940992 00000100:00100000:1.0:1787784616.730243:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657940992 00000100:00100000:1.0:1787784616.730318:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 552 00000100:00100000:1.0:1787784616.730329:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:17187:x1874627657940992:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1787784616.730433:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:17187:x1874627657940992:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 106us (1913us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784616.730455:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 552 00000100:00080000:2.0:1787784616.730675:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784616.732615:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657941248 00000100:00100000:2.0:1787784616.732624:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657941248 00000100:00100000:2.0:1787784616.732679:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 122 00000100:00100000:2.0:1787784616.732700:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:17187:x1874627657941248:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784616.733127:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+23:17187:x1874627657941248:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 439us (659us total) trans 4294967692 rc 0/0 00000100:00100000:2.0:1787784616.733136:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 122 00000001:00080000:2.0:1787784616.989189:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967691 is committed 00000001:00080000:2.0:1787784616.989215:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967692 is committed 00010000:00010000:3.0:1787784617.906176:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787784618.314932:0:15706:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 9a: Multiple remote agents ============ 18:50:16 (1787784616) 00000100:00080000:2.0:1787784618.483643:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784618.765974:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784618.766063:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784618.766091:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784618.766138:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784618.766218:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784618.766238:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784618.766287:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657941760 00000100:00100000:3.0:1787784618.766296:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657941760 00000100:00100000:3.0:1787784618.766351:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657942016 00000100:00100000:3.0:1787784618.766354:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657942016 00000100:00100000:3.0:1787784618.766372:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657941632 00000100:00100000:3.0:1787784618.766374:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657941632 00000100:00100000:3.0:1787784618.766391:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 170 00000100:00100000:3.0:1787784618.766400:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2354:x1874627657941760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784618.766467:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2354:x1874627657941760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 68us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784618.766492:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 170 00000100:00100000:3.0:1787784618.766509:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 171 00000100:00100000:3.0:1787784618.766515:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2354:x1874627657942016:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784618.766539:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2354:x1874627657942016:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (447us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784618.766545:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 171 00000100:00100000:3.0:1787784618.766558:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 172 00000100:00100000:3.0:1787784618.766564:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657941632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784618.766584:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657941632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (367us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784618.766591:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 172 00000100:00100000:3.0:1787784618.766610:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657941376 00000100:00100000:3.0:1787784618.766613:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657941376 00000100:00100000:3.0:1787784618.766636:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00100000:3.0:1787784618.766642:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657941376:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784618.766665:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657941376:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (527us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784618.766671:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00080000:3.0:1787784618.766698:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784618.766702:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784618.775490:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657941504 00000100:00100000:2.0:1787784618.775505:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657941504 00000100:00100000:1.0:1787784618.775529:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784618.775560:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657942144 00000100:00100000:1.0:1787784618.775564:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657942144 00000100:00080000:3.0:1787784618.775937:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784618.775942:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784618.778293:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784618.778342:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 235 00000100:00100000:3.0:1787784618.778354:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2354:x1874627657941888:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784618.778402:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2354:x1874627657941888:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 50us (12160us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784618.782322:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:1.0:1787784618.782338:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657942144:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784618.782411:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657942144:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (6888us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784618.782419:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:3.0:1787784618.787005:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 235 00010000:00010000:3.0:1787784619.953943:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787784620.106847:0:15754:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 00000100:00100000:2.0:1787784620.529469:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784620.529506:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a91e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668322944:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784620.529608:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1787784620.530106:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784620.530507:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668323072:0@lo:400:kworker.0 00000100:00100000:3.0:1787784620.530589:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784620.530648:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539130000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668323584:0@lo:400:kworker.0 00000100:00100000:2.0:1787784620.530734:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668323328:0@lo:400:kworker.0 00000100:00100000:3.0:1787784620.530941:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784620.531028:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668323456 00000100:00100000:3.0:1787784620.531041:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668323456 00000100:00100000:3.0:1787784620.531113:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:3.0:1787784620.531126:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668323456:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784620.531247:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668323456:12345-0@lo:400:kworker.0 Request processed in 123us (664us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784620.531256:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00080000:3.0:1787784620.531298:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784620.531308:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784620.531354:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539130f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668323456:0@lo:400:kworker.0 00000100:00100000:2.0:1787784620.531395:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784620.531401:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668323200:0@lo:400:kworker.0 00000100:00100000:3.0:1787784620.531407:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668323072:0@lo:400:kworker.0 00000100:00100000:2.0:1787784620.531468:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a91e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668322944:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1787784620.532614:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784620.533920:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539130000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668323584:0@lo:400:kworker.0 00000100:00100000:2.0:1787784620.536100:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668323328:0@lo:400:kworker.0 00000100:00100000:1.0:1787784620.538800:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784620.543656:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668323200:0@lo:400:kworker.0 00000001:02000400:2.0:1787784621.433113:0:15802:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10a: Archive a file =================== 18:50:19 (1787784619) 00000040:00080000:3.0:1787784621.617332:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784621.617340:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784621.618780:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 12 count 1 00000040:00080000:2.0:1787784621.618791:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784621.756921:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784621.757235:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657942400 00000100:00100000:2.0:1787784621.757245:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657942400 00000100:00100000:2.0:1787784621.757324:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 553 00000100:00100000:2.0:1787784621.757336:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+22:18622:x1874627657942400:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784621.757357:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784621.757375:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fa1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0586 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.757444:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.757459:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.757470:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.757536:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.757544:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.757552:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787784621.757874:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784621.758396:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3fb6 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.758410:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3fb6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.758418:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3fb6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784621.758428:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:2.0:1787784621.760252:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760266:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760274:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760280:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784621.760285:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760292:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760307:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:2.0:1787784621.760310:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.760314:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760333:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc3fa8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760348:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760354:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760360:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760365:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784621.760368:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760374:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760382:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff989519d44700) 00010000:00010000:2.0:1787784621.760385:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.760388:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760394:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3faf lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760853:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fa1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0586 expref: 25 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760884:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc3fb6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0586 expref: 24 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.760893:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784621.760895:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.760898:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401099c00, rc 0) 00000100:00100000:2.0:1787784621.760980:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18622:x1874627657942400:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3645us (4066us total) trans 4294967693 rc 0/0 00000100:00100000:2.0:1787784621.760989:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 553 00000100:00100000:1.0:1787784621.777704:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784621.779877:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784621.780031:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657942656 00000100:00100000:2.0:1787784621.780052:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657942656 00000100:00100000:2.0:1787784621.780128:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 554 00000100:00100000:2.0:1787784621.780139:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18622:x1874627657942656:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1787784621.780280:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18622:x1874627657942656:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 143us (411us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784621.780287:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 554 00000100:00100000:1.0:1787784621.794924:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:3.0:1787784621.796365:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784621.797684:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+55:18623:x1874627657942784:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1178us (2772us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784621.797703:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 555 00000100:00100000:3.0:1787784621.804626:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784621.804714:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657943040 00000100:00100000:2.0:1787784621.804740:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657943040 00000100:00100000:2.0:1787784621.804910:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 556 00000100:00100000:2.0:1787784621.804921:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+55:18623:x1874627657943040:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:2.0:1787784621.804983:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784621.805011:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc3e27 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b04fa expref: 55 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.805030:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649300) 00010000:00010000:2.0:1787784621.805062:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784621.808271:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.808299:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc3e66 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b050f expref: 22 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.808310:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.808349:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc3e66 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b050f expref: 22 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.808367:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc3e66 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b050f expref: 22 pid: 5813 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784621.808383:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668323712:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784621.808484:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784621.824357:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.825233:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657943168 00000100:00100000:3.0:1787784621.825243:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657943168 00010000:00010000:3.0:1787784621.825284:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc3e66 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b050f expref: 23 pid: 5813 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784621.825328:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 172 00000100:00100000:3.0:1787784621.825336:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2356:x1874627657943168:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784621.825351:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784621.825357:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc3e66 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b050f expref: 23 pid: 5813 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.825370:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc3e66 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b050f expref: 23 pid: 5813 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.825392:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649300) 00010000:00010000:3.0:1787784621.825397:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.825407:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.825415:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.825434:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784621.825490:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2356:x1874627657943168:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 154us (1141us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784621.825497:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 172 00010000:00010000:2.0:1787784621.831003:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc3e66 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b050f expref: 22 pid: 5813 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784621.831031:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668323712:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784621.831057:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.831068:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.831504:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.831546:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.831558:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.831569:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784621.835904:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 512 00010000:00010000:2.0:1787784621.835923:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.835933:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.835940:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.835946:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784621.835950:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.835957:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.835974:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a649300) 00010000:00010000:2.0:1787784621.835977:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.835982:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.836001:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fd9 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784621.836022:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b3346888 reply state 000000009c7894cc transno 4294967694 ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784621.836031:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784621.836169:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:18623:x1874627657943040:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 31225us (31532us total) trans 4294967694 rc 0/0 00000100:00100000:2.0:1787784621.836179:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 556 00010000:00010000:2.0:1787784621.838608:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.838625:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.838633:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.838639:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784621.838645:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.838653:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.838670:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff989519d44700) 00010000:00010000:2.0:1787784621.838673:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.838677:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.838700:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe0 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784621.869289:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784621.869460:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657943296 00000100:00100000:2.0:1787784621.869470:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657943296 00000100:00100000:2.0:1787784621.869542:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 557 00000100:00100000:2.0:1787784621.869552:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+55:18624:x1874627657943296:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:2.0:1787784621.869576:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784621.869598:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0594 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.869674:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fee lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.869688:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.869700:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784621.869770:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 7680 00000004:00200000:2.0:1787784621.869774:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 14 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784621.869796:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.869817:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc3fe7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0594 expref: 57 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.869834:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fee lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0594 expref: 56 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.869850:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44700) 00010000:00010000:2.0:1787784621.869853:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784621.869857:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098600, rc 0) 00000100:00100000:2.0:1787784621.869928:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:18624:x1874627657943296:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 378us (647us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784621.869935:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 557 00000100:00100000:1.0:1787784621.873511:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.873666:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657943424 00000100:00100000:3.0:1787784621.873677:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657943424 00000100:00100000:3.0:1787784621.873747:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 558 00000100:00100000:3.0:1787784621.873757:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:2356:x1874627657943424:12345-192.168.203.43@tcp:41:mkdir.0 00000004:00000024:3.0:1787784621.873810:0:5813:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 2259943424 avail 2257846272 00000100:00100000:3.0:1787784621.873883:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:2356:x1874627657943424:12345-192.168.203.43@tcp:41:mkdir.0 Request processed in 128us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784621.873890:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 558 00000100:00100000:1.0:1787784621.878031:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784621.878313:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657943680 00000100:00100000:2.0:1787784621.878324:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657943680 00000100:00100000:2.0:1787784621.878430:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 559 00000100:00100000:2.0:1787784621.878441:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:18624:x1874627657943680:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:2.0:1787784621.878505:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784621.878530:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc3fee lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0594 expref: 56 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.878548:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44700) 00010000:00010000:2.0:1787784621.878559:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784621.878731:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+55:18624:x1874627657943680:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 291us (709us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787784621.878748:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 559 00000100:00100000:1.0:1787784621.883534:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.883717:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657943808 00000100:00100000:3.0:1787784621.883728:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657943808 00000100:00100000:3.0:1787784621.883805:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 560 00000100:00100000:3.0:1787784621.883816:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+55:18624:x1874627657943808:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:3.0:1787784621.883835:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784621.883862:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc3ff5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b059b expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.883926:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3ffc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.883940:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3ffc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.883951:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3ffc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784621.884009:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 7680 00000004:00200000:3.0:1787784621.884013:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 14 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784621.884036:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3ffc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.884078:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc3ff5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b059b expref: 57 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.884096:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3ffc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b059b expref: 56 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.884104:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784621.884107:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784621.884110:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655a00, rc 0) 00000100:00100000:3.0:1787784621.884198:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:18624:x1874627657943808:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 383us (671us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784621.884207:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 560 00000100:00100000:1.0:1787784621.890127:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784621.890284:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657943936 00000100:00100000:0.0:1787784621.890294:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657943936 00000100:00100000:0.0:1787784621.890352:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 561 00000100:00100000:0.0:1787784621.890362:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+56:18624:x1874627657943936:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784621.890382:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784621.890400:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4003 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05a2 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890454:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890468:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890492:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890507:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890513:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890520:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890610:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4018 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890644:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4018 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890651:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4018 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784621.890689:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 512 00000004:00200000:0.0:1787784621.890693:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784621.890713:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4018 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890724:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890731:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890748:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890754:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.890759:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890766:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890778:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:0.0:1787784621.890781:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.890785:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890802:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc400a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890815:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890820:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890826:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890831:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.890834:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890840:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890847:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff98950a162200) 00010000:00010000:0.0:1787784621.890850:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.890853:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890859:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4011 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890886:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4003 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05a2 expref: 58 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890898:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4018 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05a2 expref: 57 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.890910:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff98950a162700) 00010000:00010000:0.0:1787784621.890912:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784621.890915:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:0.0:1787784621.890968:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:18624:x1874627657943936:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 606us (849us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784621.890976:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 561 00000100:00100000:1.0:1787784621.898524:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784621.898615:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657944192 00000100:00100000:2.0:1787784621.898637:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657944192 00000100:00100000:2.0:1787784621.898694:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 562 00000100:00100000:2.0:1787784621.898703:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:18624:x1874627657944192:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:2.0:1787784621.898770:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784621.898797:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4018 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05a2 expref: 57 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.898830:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff98950a162700) 00010000:00010000:2.0:1787784621.898842:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784621.899017:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.899031:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.899041:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.899091:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.899098:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.899104:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784621.899986:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x24:0x0]: returning size 512 00010000:00010000:3.0:1787784621.900000:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.900008:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.900015:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.900021:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.900026:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.900034:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.900045:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784621.900048:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.900052:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.900058:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc401f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784621.900075:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000064eeddeb reply state 000000006780f875 transno 4294967695 ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784621.900084:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784621.900207:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:18624:x1874627657944192:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 1501us (1690us total) trans 4294967695 rc 0/0 00000100:00100000:3.0:1787784621.900228:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 562 00010000:00010000:3.0:1787784621.907334:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.907353:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.907361:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.907367:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540654000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.907371:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.907379:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.907395:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x43454b14 (ffff989505542f00) 00010000:00010000:3.0:1787784621.907399:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.907404:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.907410:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc4026 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784621.908206:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.908639:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657944320 00000100:00100000:3.0:1787784621.908648:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657944320 00000100:00100000:3.0:1787784621.908714:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:3.0:1787784621.908725:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+57:18625:x1874627657944320:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784621.908866:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784621.908896:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654800/0xb0ce63137acc402d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05a9 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.908957:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.908971:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.908981:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.908997:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909003:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909009:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909116:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc4042 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909124:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc4042 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909131:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc4042 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784621.909289:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 512 00000004:00200000:3.0:1787784621.909294:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784621.909335:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc4042 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909355:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909362:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909368:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909374:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.909378:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909385:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909397:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784621.909401:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.909405:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909411:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4034 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909421:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909427:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909433:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909445:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.909448:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909454:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909462:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff989505542b00) 00010000:00010000:3.0:1787784621.909465:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.909468:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909474:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc403b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909504:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654800/0xb0ce63137acc402d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05a9 expref: 59 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909516:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655400/0xb0ce63137acc4042 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05a9 expref: 58 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.909523:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:3.0:1787784621.909525:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784621.909528:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655400, rc 0) 00000100:00100000:3.0:1787784621.909590:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:18625:x1874627657944320:12345-192.168.203.43@tcp:101:bash.0 Request processed in 866us (1388us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784621.909597:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:1.0:1787784621.912622:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.912703:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657944448 00000100:00100000:3.0:1787784621.912725:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657944448 00000100:00100000:3.0:1787784621.912928:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 564 00000100:00100000:3.0:1787784621.912939:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:18625:x1874627657944448:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784621.912962:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784621.912982:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4049 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05b0 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913059:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913073:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913086:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913100:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913108:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913114:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913209:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913216:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913223:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913235:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.913239:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913246:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913257:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:3.0:1787784621.913260:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.913264:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913269:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4050 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913282:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913288:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913295:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913300:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.913303:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913309:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913337:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff989505542600) 00010000:00010000:3.0:1787784621.913340:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.913343:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913350:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4057 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913383:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4049 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b0 expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913394:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:3.0:1787784621.913396:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.913399:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4049 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b0 expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.913409:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655c00, rc 0) 00000100:00100000:3.0:1787784621.913458:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:18625:x1874627657944448:12345-192.168.203.43@tcp:101:bash.0 Request processed in 521us (845us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784621.913465:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 564 00000100:00100000:1.0:1787784621.924545:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784621.925209:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657944576 00000100:00100000:0.0:1787784621.925219:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657944576 00000100:00100000:0.0:1787784621.925279:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 565 00000100:00100000:0.0:1787784621.925289:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:18626:x1874627657944576:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784621.925310:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784621.925339:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc405e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05b7 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.925419:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.925432:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.925445:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784621.925501:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 7680 00000004:00200000:0.0:1787784621.925505:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 14 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784621.925528:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.925562:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc405e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 24 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.925581:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 23 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.925589:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:0.0:1787784621.925592:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784621.925595:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16000, rc 0) 00000100:00100000:0.0:1787784621.925656:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+23:18626:x1874627657944576:12345-192.168.203.43@tcp:101:rm.0 Request processed in 369us (1120us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784621.925663:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 565 00000100:00100000:1.0:1787784621.932573:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.932654:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657944704 00000100:00100000:3.0:1787784621.932664:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657944704 00000100:00100000:3.0:1787784621.932744:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 566 00000100:00100000:3.0:1787784621.932754:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+23:18626:x1874627657944704:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784621.932775:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784621.932942:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc406c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05be expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933024:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933037:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933047:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933059:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933065:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933071:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933211:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc4081 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933221:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc4081 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933229:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc4081 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784621.933258:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 512 00000004:00200000:3.0:1787784621.933262:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784621.933284:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc4081 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933305:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933311:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933426:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933432:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.933436:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933444:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933456:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784621.933459:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.933464:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933470:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4073 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933484:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933490:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933495:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933508:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.933511:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933518:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933526:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff989505542600) 00010000:00010000:3.0:1787784621.933528:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.933531:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933537:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc407a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933566:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc406c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05be expref: 25 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933579:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc4081 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05be expref: 24 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.933586:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:3.0:1787784621.933588:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784621.933591:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905e00, rc 0) 00000100:00100000:2.0:1787784621.933726:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18626:x1874627657944704:12345-192.168.203.43@tcp:101:rm.0 Request processed in 972us (1160us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784621.933748:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 566 00000100:00100000:1.0:1787784621.938934:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784621.939011:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657944832 00000100:00100000:0.0:1787784621.939020:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657944832 00000100:00100000:0.0:1787784621.939073:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 567 00000100:00100000:0.0:1787784621.939083:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18626:x1874627657944832:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784621.939104:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784621.939123:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4088 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05c5 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939211:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939225:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939235:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939264:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939272:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939278:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939345:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939353:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939360:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939366:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.939370:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939377:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939388:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:0.0:1787784621.939391:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.939395:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939401:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc408f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939421:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939427:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939432:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939436:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.939439:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939445:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939452:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff98950a162200) 00010000:00010000:0.0:1787784621.939454:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.939457:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939464:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4096 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939493:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4088 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05c5 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939504:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:0.0:1787784621.939506:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.939517:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4088 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05c5 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.939527:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16400, rc 0) 00000100:00100000:0.0:1787784621.939575:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18626:x1874627657944832:12345-192.168.203.43@tcp:101:rm.0 Request processed in 494us (649us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784621.939583:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 567 00000100:00100000:1.0:1787784621.943986:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784621.944384:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657944960 00000100:00100000:0.0:1787784621.944394:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657944960 00000100:00100000:0.0:1787784621.944524:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 568 00000100:00100000:0.0:1787784621.944535:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18626:x1874627657944960:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784621.944556:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784621.944579:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc409d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05cc expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944662:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944686:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944698:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944714:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944722:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944743:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944820:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944827:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944834:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944840:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.944845:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944852:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944864:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:0.0:1787784621.944867:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.944871:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944877:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944889:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944895:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944901:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944912:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbde00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.944916:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944923:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944931:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff98950a162f00) 00010000:00010000:0.0:1787784621.944933:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.944936:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944943:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944977:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc409d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05cc expref: 25 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.944988:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:0.0:1787784621.944990:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.944994:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc409d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05cc expref: 25 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.945004:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:0.0:1787784621.945066:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18626:x1874627657944960:12345-192.168.203.43@tcp:101:rm.0 Request processed in 532us (1089us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784621.945074:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 568 00000100:00100000:1.0:1787784621.964450:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784621.964584:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657945088 00000100:00100000:0.0:1787784621.964606:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657945088 00000100:00100000:0.0:1787784621.964740:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 569 00000100:00100000:0.0:1787784621.964752:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18627:x1874627657945088:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784621.964772:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784621.964794:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05d3 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964868:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964883:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964895:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964909:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964916:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964923:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964989:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.964997:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965004:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965017:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.965021:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965028:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965039:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:0.0:1787784621.965043:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.965047:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965053:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc40b9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965067:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965073:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965078:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965083:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784621.965440:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965447:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965465:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff98950a162f00) 00010000:00010000:0.0:1787784621.965467:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.965470:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965476:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc40c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965513:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40b2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05d3 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965524:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:0.0:1787784621.965527:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784621.965530:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc40b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05d3 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.965548:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbde00, rc 0) 00000100:00100000:1.0:1787784621.965686:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18627:x1874627657945088:12345-192.168.203.43@tcp:101:cp.0 Request processed in 935us (1244us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784621.965693:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 569 00000100:00100000:1.0:1787784621.970056:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.970141:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657945216 00000100:00100000:3.0:1787784621.970223:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657945216 00000100:00100000:2.0:1787784621.970722:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 570 00000100:00100000:2.0:1787784621.970745:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508342450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18627:x1874627657945216:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784621.970764:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784621.970784:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc40c7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05da expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970845:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970860:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970870:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970884:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970891:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970897:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970975:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970982:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970988:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.970994:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784621.970998:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971005:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971016:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:2.0:1787784621.971019:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.971024:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971030:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc40ce lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971040:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971046:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971052:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971064:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784621.971068:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971074:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971082:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff989519d44700) 00010000:00010000:2.0:1787784621.971085:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.971088:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971094:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc40d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971129:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc40c7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05da expref: 25 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971139:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:2.0:1787784621.971142:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.971386:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc40c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05da expref: 25 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.971399:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098600, rc 0) 00000100:00100000:2.0:1787784621.971512:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508342450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18627:x1874627657945216:12345-192.168.203.43@tcp:101:cp.0 Request processed in 779us (1465us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784621.971521:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 570 00000100:00100000:1.0:1787784621.979018:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.979212:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657945472 00000100:00100000:3.0:1787784621.979222:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657945472 00000100:00100000:3.0:1787784621.979377:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 571 00000100:00100000:3.0:1787784621.979410:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18627:x1874627657945472:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784621.979439:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784621.979459:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc40dc lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05e1 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.979580:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.979594:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.979606:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.979619:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 3/0,1 mode: --/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.979627:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.979633:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784621.985595:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xc:0x0], index 0: 0 00010000:00010000:3.0:1787784621.989689:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc40f1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.989944:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc40f1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.989959:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc40f1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990030:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990038:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990116:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990123:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784621.990127:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990135:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990211:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:3.0:1787784621.990215:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784621.990221:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990227:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc40e3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784621.990243:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007cf36ab0 reply state 000000004075a284 transno 4294967696 ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784621.990253:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787784621.990524:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc40dc lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e1 expref: 27 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990555:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904600/0xb0ce63137acc40f1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e1 expref: 26 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.990564:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:3.0:1787784621.990567:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784621.990569:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904600, rc 0) 00000100:00100000:3.0:1787784621.990708:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+27:18627:x1874627657945472:12345-192.168.203.43@tcp:101:cp.0 Request processed in 11300us (11696us total) trans 4294967696 rc 0/0 00000100:00100000:3.0:1787784621.990738:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 571 00000100:00100000:1.0:1787784621.994506:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:2.0:1787784621.994644:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.994671:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.994676:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.994680:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784621.994684:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.994689:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.994702:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff989505542600) 00010000:00010000:2.0:1787784621.994704:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784621.994707:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 2/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784621.994711:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc40ea lrc: 0/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784621.994841:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657945600 00000100:00100000:0.0:1787784621.994851:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657945600 00000100:00100000:0.0:1787784621.994951:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 145 00000100:00100000:0.0:1787784621.994962:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+6:2356:x1874627657945600:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784621.994978:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784621.994993:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05e8 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784621.995017:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784621.995036:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=413s v=(0 0) 00010000:00010000:0.0:1787784621.995047:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784621.995056:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 7 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.995076:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 7 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784621.995267:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbde00, rc 0) 00000100:00080000:3.0:1787784621.995363:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784621.995498:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657945600:12345-192.168.203.43@tcp:101:cp.0 Request processed in 536us (998us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784621.995505:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 145 00000100:00100000:1.0:1787784621.998799:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784621.998917:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657945728 00000100:00100000:3.0:1787784621.998926:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657945728 00000100:00100000:3.0:1787784621.999024:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 146 00000100:00100000:3.0:1787784621.999035:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+7:18627:x1874627657945728:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784621.999052:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784621.999069:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05ef expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.999109:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 8 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.999118:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784621.999122:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784621.999126:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 8 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.999210:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 8 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784621.999224:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 8 pid: 6669 timeout: 513 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784621.999239:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668323840:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784621.999315:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989505495600 going to sleep for 11 seconds 00010000:00010000:1.0:1787784622.001662:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784622.002764:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784622.002836:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657945856 00000100:00100000:1.0:1787784622.002844:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657945856 00010000:00010000:1.0:1787784622.002877:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 9 pid: 6669 timeout: 513 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784622.002931:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:1.0:1787784622.002939:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657945856:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784622.002952:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784622.002971:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 9 pid: 6669 timeout: 513 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.002983:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 9 pid: 6669 timeout: 513 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.003005:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784622.003009:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784622.003017:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.003027:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.003059:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784622.003071:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668323968:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784622.003171:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677600 going to sleep for 11 seconds 00010000:00010000:3.0:1787784622.003317:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000476f99e8/0xb0ce63137acc40ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05e8 expref: 9 pid: 6669 timeout: 513 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784622.003331:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668323840:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784622.003353:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 8 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.003363:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905200, rc 0) 00000100:00100000:3.0:1787784622.003396:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:18627:x1874627657945728:12345-192.168.203.43@tcp:101:cp.0 Request processed in 4364us (4606us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784622.003404:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 146 00000100:00100000:2.0:1787784622.007017:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668323968:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784622.007041:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784622.007097:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2357:x1874627657945856:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 4158us (4338us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784622.007108:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:2.0:1787784622.009966:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784622.010061:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657946112 00000100:00100000:0.0:1787784622.010071:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657946112 00000100:00100000:0.0:1787784622.010134:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 123 00000100:00100000:0.0:1787784622.010144:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18627:x1874627657946112:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:0.0:1787784622.010517:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+27:18627:x1874627657946112:12345-192.168.203.43@tcp:35:cp.0 Request processed in 373us (558us total) trans 4294967697 rc 0/0 00000100:00100000:0.0:1787784622.010526:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 123 00000100:00100000:1.0:1787784622.051667:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784622.051784:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657946240 00000100:00100000:0.0:1787784622.051794:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657946240 00000100:00100000:0.0:1787784622.051856:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 572 00000100:00100000:0.0:1787784622.051867:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:18630:x1874627657946240:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784622.051887:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784622.051907:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc410d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05f6 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.051975:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.051988:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052013:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052028:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052035:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052041:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052134:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc200/0xb0ce63137acc4122 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052143:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc200/0xb0ce63137acc4122 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052188:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc200/0xb0ce63137acc4122 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784622.052220:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00000004:00200000:0.0:1787784622.052224:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784622.052241:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc200/0xb0ce63137acc4122 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052253:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc200/0xb0ce63137acc4122 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052260:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052277:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052283:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052289:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784622.052293:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052300:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052312:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:0.0:1787784622.052315:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784622.052319:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052325:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4114 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052338:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052343:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052349:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052354:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784622.052362:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052369:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052376:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff98950a162d00) 00010000:00010000:0.0:1787784622.052378:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784622.052381:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052495:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc411b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052528:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc410d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05f6 expref: 60 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052542:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc200/0xb0ce63137acc4122 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05f6 expref: 59 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.052549:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:0.0:1787784622.052551:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784622.052554:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc200, rc 0) 00000100:00100000:1.0:1787784622.053288:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:18630:x1874627657946240:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1418us (1626us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784622.053307:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 572 00000100:00100000:1.0:1787784622.058053:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.058119:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657946368 00000100:00100000:3.0:1787784622.058129:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657946368 00000100:00100000:3.0:1787784622.058234:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 147 00000100:00100000:3.0:1787784622.058244:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657946368:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784622.058281:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784622.058299:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4129 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b05fd expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784622.058370:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668324096:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784622.058463:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989505495600 going to sleep for 11 seconds 00000100:00080000:2.0:1787784622.059677:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1787784622.061015:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1787784622.061023:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb mtime: 0 -> 1787784621 00002000:00010000:3.0:1787784622.061025:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb atime: 0 -> 1787784621 00002000:00010000:3.0:1787784622.061027:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb ctime: 0 -> 1787784621 00000100:00100000:3.0:1787784622.061063:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668324096:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784622.061100:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4129 lrc: 1/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05fd expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.061124:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4129 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05fd expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.061137:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904c00, rc 0) 00000100:00100000:3.0:1787784622.061218:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657946368:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2975us (3175us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784622.061227:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 147 00000100:00100000:0.0:1787784622.064941:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784622.069023:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657946496 00000100:00100000:1.0:1787784622.069036:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657946496 00000100:00100000:1.0:1787784622.069182:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:1.0:1787784622.069195:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:18630:x1874627657946496:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784622.069215:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784622.069235:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4130 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0604 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.069371:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ada00/0xb0ce63137acc4137 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.069385:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ada00/0xb0ce63137acc4137 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.069397:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ada00/0xb0ce63137acc4137 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784622.069441:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 512 00000004:00200000:1.0:1787784622.069445:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784622.069465:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ada00/0xb0ce63137acc4137 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.069491:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4130 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0604 expref: 61 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.069508:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ada00/0xb0ce63137acc4137 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0604 expref: 60 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.069532:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:1.0:1787784622.069535:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784622.069538:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ada00, rc 0) 00000100:00100000:1.0:1787784622.069597:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:18630:x1874627657946496:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 405us (4664us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784622.069603:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:0.0:1787784622.081891:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784622.081928:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657946624 00000100:00100000:0.0:1787784622.081936:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657946624 00000100:00100000:0.0:1787784622.081984:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 148 00000100:00100000:0.0:1787784622.081993:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2356:x1874627657946624:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784622.082010:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784622.082031:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc413e lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b060b expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784622.082100:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668324224:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784622.082193:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784622.088461:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668324224:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784622.088503:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc413e lrc: 1/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b060b expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.088528:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc413e lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b060b expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.088541:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd600, rc 0) 00000100:00100000:1.0:1787784622.088601:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2356:x1874627657946624:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6608us (6719us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784622.088609:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 148 00000100:00100000:2.0:1787784622.096466:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.096544:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657946880 00000100:00100000:3.0:1787784622.096553:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657946880 00000100:00100000:3.0:1787784622.096612:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 574 00000100:00100000:3.0:1787784622.096622:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:18630:x1874627657946880:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784622.097064:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].13, 136 off9960 00000100:00100000:3.0:1787784622.097193:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:18630:x1874627657946880:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 570us (736us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784622.097202:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 574 00000001:00080000:0.0:1787784622.128026:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967697 is committed 00000001:00080000:0.0:1787784622.128245:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967694 is committed 00000001:00080000:0.0:1787784622.128394:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967695 is committed 00000040:00080000:3.0:1787784622.642026:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784622.642036:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784622.642073:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 14 count 2 00000040:00080000:3.0:1787784622.642083:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784622.642699:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402aded00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668324352:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784622.642809:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953056bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784622.651188:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402aded00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668324352:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784622.659132:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784622.665282:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657947008 00000100:00100000:1.0:1787784622.665296:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657947008 00000100:00100000:1.0:1787784622.665863:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 575 00000100:00100000:1.0:1787784622.665878:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18640:x1874627657947008:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784622.665911:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xe:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784622.666033:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18640:x1874627657947008:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 160us (6911us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784622.666042:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 575 00000100:00100000:3.0:1787784622.671007:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784622.671352:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657947264 00000100:00100000:2.0:1787784622.671361:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657947264 00000100:00100000:2.0:1787784622.671411:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 576 00000100:00100000:2.0:1787784622.671421:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18640:x1874627657947264:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784622.671441:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784622.671461:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4145 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0612 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784622.671557:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc414c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784622.671571:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc414c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784622.671582:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc414c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784622.671591:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00010000:00010000:2.0:1787784622.673272:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4145 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0612 expref: 27 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784622.673301:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc414c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0612 expref: 26 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784622.673375:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:2.0:1787784622.673379:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784622.673383:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098000, rc 0) 00000100:00100000:3.0:1787784622.674352:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18640:x1874627657947264:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2926us (3348us total) trans 4294967698 rc 0/0 00000100:00100000:3.0:1787784622.674365:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 576 00000100:00100000:0.0:1787784622.685949:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784622.686000:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784622.686034:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657947520 00000100:00100000:0.0:1787784622.686044:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657947520 00000100:00100000:0.0:1787784622.686094:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 149 00000100:00100000:0.0:1787784622.686104:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657947520:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784622.686162:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.686194:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 7 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.686204:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784622.686209:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784622.686215:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 4/1,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.686249:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 7 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.686275:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 7 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784622.686291:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539131a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668324480:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784622.686367:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784622.689881:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657947392 00000100:00100000:3.0:1787784622.689896:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657947392 00000100:00100000:3.0:1787784622.689979:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 19 00000100:00100000:3.0:1787784622.689988:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2357:x1874627657947392:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784622.690003:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784622.690029:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc414c lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0612 expref: 26 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.690044:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:3.0:1787784622.690048:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784622.690108:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2357:x1874627657947392:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 121us (4107us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784622.690122:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 19 00000100:00100000:0.0:1787784622.714532:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784622.715283:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657947648 00000100:00100000:2.0:1787784622.715294:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657947648 00002000:00100000:2.0:1787784622.715426:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xc:0x0] (0->157) req@ffff989539132d00 x1874627657947648/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:503/0 lens 648/0 e 0 to 0 dl 1787784633 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784622.715437:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539132d00 with x1874627657947648 ext(0->157) 00010000:00010000:2.0:1787784622.715447:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 8 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784622.715463:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 8 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784622.715471:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989539132d00 00000100:00100000:2.0:1787784622.715499:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 3 00000100:00100000:2.0:1787784622.715508:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539132d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627657947648:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:3.0:1787784622.716455:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539131a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668324480:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784622.716482:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784622.717885:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539132d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657947648:12345-192.168.203.43@tcp:4:cp.0 Request processed in 2371us (3355us total) trans 4294967324 rc 0/0 00000100:00100000:2.0:1787784622.717898:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 3 00002000:00100000:2.0:1787784622.717910:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xc:0x0] (0->157) req@ffff989539132d00 x1874627657947648/t4294967324(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:503/0 lens 648/448 e 0 to 0 dl 1787784633 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784622.717920:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539132d00 with x1874627657947648 ext(0->157) 00010000:00010000:2.0:1787784622.717929:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 10 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784622.717945:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 10 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784622.717953:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989539132d00 00000100:00100000:3.0:1787784622.723910:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784622.724236:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657947776 00000100:00100000:0.0:1787784622.724366:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657947776 00010000:00010000:0.0:1787784622.724409:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 10 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784622.724623:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 174 00000100:00100000:0.0:1787784622.724633:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657947776:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784622.724650:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784622.724672:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 10 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.724687:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 10 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.724711:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009639c145/0xb0ce63137acc4106 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05ef expref: 10 pid: 6671 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.724735:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784622.724747:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784622.724753:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.724763:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.724801:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784622.724855:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657947776:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 224us (953us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784622.724863:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 174 00010000:00010000:1.0:1787784622.725992:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726010:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726041:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726049:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726058:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726066:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784622.726070:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726078:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726094:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.726115:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4054f17/0xb0ce63137acc415a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784622.726673:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657947520:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 40564us (40728us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784622.726685:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 149 00000100:00100000:0.0:1787784622.735400:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.735520:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657948032 00000100:00100000:3.0:1787784622.735530:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657948032 00000100:00100000:3.0:1787784622.735591:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 577 00000100:00100000:3.0:1787784622.735602:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:18640:x1874627657948032:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784622.735715:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:18640:x1874627657948032:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 115us (323us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784622.735723:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 577 00000100:00100000:0.0:1787784622.740444:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784622.740485:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657948160 00000100:00100000:0.0:1787784622.740495:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657948160 00000100:00100000:0.0:1787784622.740547:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 150 00000100:00100000:0.0:1787784622.740557:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657948160:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784622.740574:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784622.740594:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000619ac5a/0xb0ce63137acc4161 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0619 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784622.740621:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xc:0x0] initial LVB size: 158, mtime: 0x6a8f6dad, atime: 0x0, ctime: 0x6a8f6dad, blocks: 0x1 00010000:00010000:0.0:1787784622.740641:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=413s v=(0 0) 00010000:00010000:0.0:1787784622.740645:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784622.740654:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000619ac5a/0xb0ce63137acc4161 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0619 expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.740683:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000619ac5a/0xb0ce63137acc4161 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0619 expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784622.740696:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc000, rc 0) 00000100:00100000:0.0:1787784622.740757:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657948160:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 202us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784622.740764:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 150 00000100:00100000:0.0:1787784622.752044:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000040:00080000:3.0:1787784622.753622:0:15854:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784622.753629:0:15854:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784622.754052:0:15854:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 14 count 2 00000040:00080000:3.0:1787784622.754062:0:15854:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784622.755432:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657948416 00000100:00100000:1.0:1787784622.755444:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657948416 00000100:00100000:1.0:1787784622.755508:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 578 00000100:00100000:1.0:1787784622.755519:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:18640:x1874627657948416:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784622.755626:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:18640:x1874627657948416:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 109us (3592us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784622.755634:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 578 00000100:00100000:0.0:1787784622.764758:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.765666:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657948544 00000100:00100000:3.0:1787784622.765677:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657948544 00002000:00100000:3.0:1787784622.765740:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xc:0x0] (0->4095) req@ffff989539130000 x1874627657948544/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:503/0 lens 488/0 e 0 to 0 dl 1787784633 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784622.765751:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539130000 with x1874627657948544 ext(0->4095) 00002000:00010000:3.0:1787784622.765756:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989539130000 00000100:00100000:3.0:1787784622.765778:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 52 00000100:00100000:3.0:1787784622.765788:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627657948544:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784622.765998:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627657948544:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 210us (1245us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784622.766006:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 52 00002000:00100000:3.0:1787784622.766012:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xc:0x0] (0->4095) req@ffff989539130000 x1874627657948544/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:503/0 lens 488/4536 e 0 to 0 dl 1787784633 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784622.766021:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539130000 with x1874627657948544 ext(0->4095) 00002000:00010000:3.0:1787784622.766024:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989539130000 00000100:00100000:0.0:1787784622.776139:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.776280:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657948672 00000100:00100000:3.0:1787784622.776290:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657948672 00000100:00100000:3.0:1787784622.776400:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 579 00000100:00100000:3.0:1787784622.776411:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:18640:x1874627657948672:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784622.776429:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784622.776447:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc4168 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0620 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.776482:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.776495:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.776507:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.776599:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc4168 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 27 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.776630:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.776638:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:3.0:1787784622.776641:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784622.776645:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905200, rc 0) 00000100:00100000:3.0:1787784622.777301:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18640:x1874627657948672:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 890us (1166us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784622.777310:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 579 00000100:00100000:0.0:1787784622.798411:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784622.798494:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657948800 00000100:00100000:1.0:1787784622.798508:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657948800 00000100:00100000:1.0:1787784622.798568:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 580 00000100:00100000:1.0:1787784622.798578:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18640:x1874627657948800:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784622.798608:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xe:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784622.798714:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18640:x1874627657948800:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 138us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784622.798734:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 580 00000100:00100000:1.0:1787784622.810988:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.811104:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657948928 00000100:00100000:3.0:1787784622.811113:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657948928 00000100:00100000:3.0:1787784622.811195:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 151 00000100:00100000:3.0:1787784622.811206:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657948928:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784622.811248:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811266:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784622.811270:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784622.811276:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811290:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811319:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811338:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811346:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811353:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508904c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784622.811358:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811367:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811383:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.811390:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000549f1a7a/0xb0ce63137acc4176 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784622.811461:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657948928:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 258us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784622.811469:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 151 00000100:00100000:0.0:1787784622.819183:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784622.819489:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657949184 00000100:00100000:1.0:1787784622.819499:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657949184 00000100:00100000:1.0:1787784622.819564:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 581 00000100:00100000:1.0:1787784622.819575:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18640:x1874627657949184:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784622.819648:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.819666:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.819675:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 4/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.819720:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 26 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.819749:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 26 pid: 5812 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784622.819763:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bed00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668324608:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784622.819857:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508949a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784622.824892:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bed00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668324608:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784622.825014:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784622.825342:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.830553:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657949312 00000100:00100000:3.0:1787784622.830567:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657949312 00010000:00010000:3.0:1787784622.830609:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 27 pid: 5812 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784622.830699:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 175 00000100:00100000:3.0:1787784622.830709:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+27:2357:x1874627657949312:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784622.830722:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784622.830739:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 27 pid: 5812 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.830753:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 27 pid: 5812 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.830775:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc416f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0620 expref: 27 pid: 5812 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.830791:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:3.0:1787784622.830798:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.830807:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.830815:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784622.830848:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784622.830905:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784622.830915:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2357:x1874627657949312:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 206us (5573us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784622.830925:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 175 00010000:00010000:1.0:1787784622.830926:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.831116:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.831125:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.831131:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.831137:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010aca00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784622.831141:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.831177:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.831187:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:1.0:1787784622.831190:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784622.831194:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 2/0,0 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784622.831199:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc417d lrc: 0/0,0 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784622.831381:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18640:x1874627657949184:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 11740us (12141us total) trans 4294967699 rc 0/0 00000100:00100000:1.0:1787784622.831391:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 581 00000040:00080000:3.0:1787784622.831608:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784622.831619:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784622.847860:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:3.0:1787784622.847877:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784622.849908:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784622.856543:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657949568 00000100:00100000:3.0:1787784622.856561:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657949568 00000100:00100000:3.0:1787784622.857368:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 124 00000100:00100000:3.0:1787784622.857379:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+26:18640:x1874627657949568:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784622.857656:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+27:18640:x1874627657949568:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 267us (7745us total) trans 4294967700 rc 0/0 00000100:00100000:3.0:1787784622.857666:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 124 00010000:00010000:0.0:1787784623.025703:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784623.601402:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539130780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668324736:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784623.601473:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784623.601623:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668324736 00000100:00100000:0.0:1787784623.601631:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668324736 00000100:00100000:0.0:1787784623.601693:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:0.0:1787784623.601703:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668324736:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784623.601734:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 1703936 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784623.601787:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668324736:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784623.601795:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:3.0:1787784623.601850:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539130780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668324736:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1787784623.602470:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784623.893096:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784623.893324:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657949696 00000100:00100000:2.0:1787784623.893333:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657949696 00000100:00100000:2.0:1787784623.893385:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 333 00000100:00100000:2.0:1787784623.893395:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627657949696:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784623.893471:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627657949696:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 76us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784623.893478:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 333 00000100:00100000:2.0:1787784623.894661:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784623.894763:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657949824 00000100:00100000:1.0:1787784623.894771:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657949824 00000100:00100000:1.0:1787784623.894837:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00100000:1.0:1787784623.894848:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:2356:x1874627657949824:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784623.894919:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:2356:x1874627657949824:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 74us (263us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784623.894934:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00080000:2.0:1787784623.895458:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784623.897814:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784623.900614:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657949952 00000100:00100000:1.0:1787784623.900628:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657949952 00000100:00100000:1.0:1787784623.900708:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 175 00000100:00100000:1.0:1787784623.900733:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657949952:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784623.900820:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657949952:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 98us (3013us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784623.900828:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 175 00000100:00100000:0.0:1787784623.905770:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784623.905823:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784623.905915:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657950080 00000100:00100000:2.0:1787784623.905924:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657950080 00000100:00100000:2.0:1787784623.906011:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 176 00000100:00100000:2.0:1787784623.906022:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2355:x1874627657950080:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784623.906092:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2355:x1874627657950080:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 72us (332us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784623.906099:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 176 00000100:00080000:0.0:1787784623.906256:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784623.908652:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657950208 00000100:00100000:3.0:1787784623.908666:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657950208 00000100:00100000:3.0:1787784623.908760:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:3.0:1787784623.908770:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+27:2355:x1874627657950208:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784623.908846:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+27:2355:x1874627657950208:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (3022us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784623.908854:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00080000:0.0:1787784623.909141:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784623.915440:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784623.915544:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784623.915744:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657950336 00000100:00100000:1.0:1787784623.915760:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657950336 00000100:00100000:1.0:1787784623.915854:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657950464 00000100:00100000:1.0:1787784623.915858:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657950464 00000100:00100000:1.0:1787784623.915892:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:1.0:1787784623.915911:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627657950336:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784623.916001:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627657950336:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 91us (576us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784623.916008:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:1.0:1787784623.916025:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:1.0:1787784623.916031:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657950464:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784623.916074:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657950464:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 43us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784623.916096:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00080000:0.0:1787784623.917180:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784623.917186:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784624.050352:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784624.698275:0:15905:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784624.698285:0:15905:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784624.698321:0:15905:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 14 count 2 00000040:00080000:0.0:1787784624.698327:0:15905:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784624.992117:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784624.992239:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657950592 00000100:00100000:1.0:1787784624.992249:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657950592 00000100:00100000:1.0:1787784624.992324:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 152 00000100:00100000:1.0:1787784624.992334:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657950592:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:1.0:1787784624.992361:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784624.992379:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7e505dd/0xb0ce63137acc4184 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0627 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784624.992405:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=3s v=(0 0) 00010000:00010000:1.0:1787784624.992410:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784624.992419:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7e505dd/0xb0ce63137acc4184 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0627 expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784624.992448:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7e505dd/0xb0ce63137acc4184 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0627 expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784624.992462:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac400, rc 0) 00000100:00100000:1.0:1787784624.992519:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627657950592:12345-192.168.203.43@tcp:101:diff.0 Request processed in 187us (411us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784624.992527:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 152 00000100:00080000:0.0:1787784624.993292:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784625.007866:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784625.008014:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657950848 00000100:00100000:2.0:1787784625.008024:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657950848 00000100:00100000:2.0:1787784625.008093:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 582 00000100:00100000:2.0:1787784625.008103:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+58:18667:x1874627657950848:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:2.0:1787784625.008123:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784625.008203:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc418b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b062e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784625.008316:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4192 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784625.008330:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4192 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784625.008344:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4192 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784625.008364:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00010000:00010000:2.0:1787784625.008510:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc418b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b062e expref: 61 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784625.008528:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4192 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b062e expref: 60 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784625.008536:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:2.0:1787784625.008539:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784625.008542:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098e00, rc 0) 00000100:00100000:2.0:1787784625.008596:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:18667:x1874627657950848:12345-192.168.203.43@tcp:101:diff.0 Request processed in 495us (740us total) trans 4294967701 rc 0/0 00000100:00100000:2.0:1787784625.008603:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 582 00000100:00080000:0.0:1787784625.009735:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784625.019668:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784625.019783:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784625.022741:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657951104 00000100:00100000:1.0:1787784625.022757:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657951104 00002000:00100000:1.0:1787784625.022824:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xc:0x0] (0->4095) req@ffff98951b45f840 x1874627657951104/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:506/0 lens 488/0 e 0 to 0 dl 1787784636 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784625.022854:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b45f840 with x1874627657951104 ext(0->4095) 00002000:00010000:1.0:1787784625.022860:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b45f840 00000100:00100000:1.0:1787784625.022905:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 53 00000100:00100000:1.0:1787784625.022915:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657951104:12345-192.168.203.43@tcp:3:diff.0 00000100:00100000:1.0:1787784625.023124:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657951104:12345-192.168.203.43@tcp:3:diff.0 Request processed in 209us (3340us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787784625.023132:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 53 00002000:00100000:1.0:1787784625.023139:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xc:0x0] (0->4095) req@ffff98951b45f840 x1874627657951104/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:506/0 lens 488/4536 e 0 to 0 dl 1787784636 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784625.023182:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b45f840 with x1874627657951104 ext(0->4095) 00002000:00010000:1.0:1787784625.023185:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b45f840 00000100:00100000:0.0:1787784625.025699:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657950976 00000100:00100000:0.0:1787784625.025742:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657950976 00000100:00100000:0.0:1787784625.025822:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 20 00000100:00100000:0.0:1787784625.025833:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:2356:x1874627657950976:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784625.025848:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784625.025874:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4192 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b062e expref: 60 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784625.025894:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:0.0:1787784625.025898:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784625.025971:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:2356:x1874627657950976:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 140us (6313us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784625.025986:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 20 00000100:00100000:2.0:1787784625.032139:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784625.032253:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657951360 00000100:00100000:3.0:1787784625.032262:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657951360 00000100:00100000:3.0:1787784625.032327:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 125 00000100:00100000:3.0:1787784625.032338:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950767a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:18667:x1874627657951360:12345-192.168.203.43@tcp:35:diff.0 00000100:00100000:3.0:1787784625.032566:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950767a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:18667:x1874627657951360:12345-192.168.203.43@tcp:35:diff.0 Request processed in 229us (436us total) trans 4294967702 rc 0/0 00000100:00100000:3.0:1787784625.033062:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 125 00000100:00100000:3.0:1787784625.072303:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784625.072369:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657951616 00000100:00100000:2.0:1787784625.072381:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657951616 00000100:00100000:3.0:1787784625.072583:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 583 00000100:00100000:3.0:1787784625.072596:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:18670:x1874627657951616:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784625.072620:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784625.072639:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc41a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0635 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.072797:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc41a7 lrc: 3/1,0 mode: --/CR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.072811:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc41a7 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.072823:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc41a7 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784625.072833:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00010000:00010000:3.0:1787784625.072944:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc41a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0635 expref: 63 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.072962:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508904c00/0xb0ce63137acc41a7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0635 expref: 62 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.072970:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:3.0:1787784625.072973:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1787784625.072976:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508904c00, rc 0) 00000100:00100000:3.0:1787784625.073070:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:18670:x1874627657951616:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 478us (774us total) trans 4294967703 rc 0/0 00000100:00100000:3.0:1787784625.073081:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 583 00010000:00010000:0.0:1787784625.073389:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784625.076967:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784625.077062:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657951744 00000100:00100000:3.0:1787784625.077073:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657951744 00000100:00100000:3.0:1787784625.077126:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00100000:3.0:1787784625.077137:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:18670:x1874627657951744:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:3.0:1787784625.077282:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077300:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077313:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077338:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077345:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077352:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077358:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784625.077363:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077370:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077382:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:3.0:1787784625.077397:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784625.077402:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784625.077408:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc41b5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784625.077482:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:18670:x1874627657951744:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 347us (524us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784625.077491:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00100000:2.0:1787784625.128400:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784625.128494:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657951872 00000100:00100000:1.0:1787784625.128504:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657951872 00000100:00100000:1.0:1787784625.128603:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 585 00000100:00100000:1.0:1787784625.128614:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:18677:x1874627657951872:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:1.0:1787784625.128681:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128697:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128720:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128747:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128766:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128773:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128779:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784625.128783:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128790:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128802:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff989505542b00) 00010000:00010000:1.0:1787784625.128805:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784625.128810:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 2/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784625.128817:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc41bc lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784625.128929:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:18677:x1874627657951872:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 317us (538us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784625.128938:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 585 00000100:00100000:2.0:1787784625.649374:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668325120:0@lo:400:kworker.0 00000100:00100000:0.0:1787784625.649402:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668324864:0@lo:400:kworker.0 00000100:00100000:2.0:1787784625.649450:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784625.649460:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784625.649554:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668324992:0@lo:400:kworker.0 00000100:00100000:3.0:1787784625.649584:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784625.649642:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784625.649648:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668325248:0@lo:400:kworker.0 00000100:00100000:2.0:1787784625.649677:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784625.649891:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668324864 00000100:00100000:1.0:1787784625.649898:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668324864 00000100:00100000:1.0:1787784625.649940:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1787784625.649948:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668324864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784625.649997:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668324864:12345-0@lo:400:kworker.0 Request processed in 49us (537us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784625.650018:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1787784625.650084:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668325248 00000100:00100000:1.0:1787784625.650089:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668325248 00000100:00100000:1.0:1787784625.650122:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1787784625.650128:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668325248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784625.650190:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668325248:12345-0@lo:400:kworker.0 Request processed in 62us (512us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784625.650196:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1787784625.650223:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784625.650231:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668325632:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784625.650258:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784625.650305:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668325120 00000100:00100000:1.0:1787784625.650309:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668325120 00000100:00100000:1.0:1787784625.650332:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668324992 00000100:00100000:1.0:1787784625.650335:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668324992 00000100:00100000:1.0:1787784625.650352:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:1.0:1787784625.650359:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668325120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784625.650389:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668325120:12345-0@lo:400:kworker.0 Request processed in 31us (941us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784625.650395:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:1.0:1787784625.650405:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:1.0:1787784625.650412:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668324992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784625.650435:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668324992:12345-0@lo:400:kworker.0 Request processed in 25us (852us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784625.650440:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00080000:1.0:1787784625.650464:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784625.650474:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784625.650489:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668325632 00000100:00100000:1.0:1787784625.650493:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668325632 00000100:00100000:1.0:1787784625.650523:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:1.0:1787784625.650530:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668325632:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784625.650555:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784625.650590:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668325632:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784625.650596:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:1.0:1787784625.650633:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668325632:0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1787784625.650653:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784625.650849:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668325248:0@lo:400:kworker.0 00000100:00100000:2.0:1787784625.650860:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668325120:0@lo:400:kworker.0 00000100:00100000:0.0:1787784625.651471:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668324864:0@lo:400:kworker.0 00000100:00100000:0.0:1787784625.651479:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391003c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668325376:0@lo:400:kworker.0 00000100:00100000:0.0:1787784625.651510:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784625.655000:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668324992:0@lo:400:kworker.0 00000100:00100000:3.0:1787784625.655016:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539103c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668325504:0@lo:400:kworker.0 00000100:00100000:3.0:1787784625.655311:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784625.661048:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668325376 00000100:00100000:1.0:1787784625.661064:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668325376 00000100:00100000:1.0:1787784625.661185:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:1.0:1787784625.661196:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668325376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784625.661300:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668325376:12345-0@lo:400:kworker.0 Request processed in 106us (9789us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784625.661307:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:3.0:1787784625.661370:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391003c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668325376:0@lo:400:kworker.0 00000100:00080000:0.0:1787784625.661867:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784625.662200:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668325504 00000100:00100000:2.0:1787784625.662272:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668325504 00000100:00100000:2.0:1787784625.662371:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1787784625.662381:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668325504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784625.662445:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668325504:12345-0@lo:400:kworker.0 Request processed in 67us (7135us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784625.662452:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1787784625.662515:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539103c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668325504:0@lo:400:kworker.0 00010000:00010000:2.0:1787784626.109068:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787784626.732065:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967324 is committed 00010000:00010000:0.0:1787784627.123493:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784627.249402:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967699 is committed 00000001:00080000:1.0:1787784627.249417:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967701 is committed 00000001:00080000:1.0:1787784627.249425:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967703 is committed 00000001:00080000:1.0:1787784627.249432:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967700 is committed 00010000:00010000:2.0:1787784628.145276:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784628.721486:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784628.721510:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668325760:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784628.721576:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784628.721699:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668325760 00000100:00100000:0.0:1787784628.721707:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668325760 00000100:00100000:0.0:1787784628.721780:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 108 00000100:00100000:0.0:1787784628.721789:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668325760:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784628.721826:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784628.721869:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668325760:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784628.721876:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 108 00000100:00080000:2.0:1787784628.722095:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784628.722128:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668325760:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784629.009693:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784629.009734:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784629.009765:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784629.009813:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784629.009832:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784629.009847:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784629.009883:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784629.009963:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952256 00000100:00100000:1.0:1787784629.009972:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952256 00000100:00100000:1.0:1787784629.010044:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952768 00000100:00100000:1.0:1787784629.010048:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952768 00000100:00100000:1.0:1787784629.010063:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952384 00000100:00100000:1.0:1787784629.010065:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952384 00000100:00100000:1.0:1787784629.010077:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952640 00000100:00100000:1.0:1787784629.010086:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952640 00000100:00100000:1.0:1787784629.010107:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:1.0:1787784629.010117:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657952256:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784629.010214:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657952256:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 98us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784629.010451:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:1.0:1787784629.010470:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:1.0:1787784629.010489:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657952768:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784629.010518:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657952768:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (706us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784629.010524:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:1.0:1787784629.010535:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 183 00000100:00100000:1.0:1787784629.010541:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2356:x1874627657952384:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784629.010561:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2356:x1874627657952384:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (714us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784629.010567:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 183 00000100:00100000:1.0:1787784629.010577:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 184 00000100:00100000:1.0:1787784629.010582:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657952640:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784629.010602:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657952640:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (721us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784629.010607:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 184 00000100:00100000:1.0:1787784629.010629:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952512 00000100:00100000:1.0:1787784629.010632:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952512 00000100:00100000:1.0:1787784629.010656:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952128 00000100:00100000:1.0:1787784629.010666:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952128 00000100:00100000:1.0:1787784629.010699:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 244 00000100:00100000:1.0:1787784629.010705:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2356:x1874627657952512:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784629.010734:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2356:x1874627657952512:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (968us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784629.010741:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 244 00000100:00100000:1.0:1787784629.010752:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:1.0:1787784629.010758:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:2356:x1874627657952128:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784629.010783:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:2356:x1874627657952128:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (950us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784629.010789:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00080000:2.0:1787784629.011421:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784629.011428:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784629.011430:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784629.011488:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952000 00000100:00100000:0.0:1787784629.011494:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952000 00000100:00100000:0.0:1787784629.011535:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:0.0:1787784629.011543:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657952000:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784629.011591:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657952000:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (1916us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784629.011598:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 335 00010000:00010000:2.0:1787784629.171475:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784629.790272:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784629.790480:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657952896 00000100:00100000:1.0:1787784629.790490:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657952896 00000100:00100000:1.0:1787784629.790591:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 586 00000100:00100000:1.0:1787784629.790601:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18622:x1874627657952896:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1787784629.790720:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18622:x1874627657952896:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 120us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784629.790729:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 586 00000100:00100000:2.0:1787784629.810939:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784629.811023:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657953152 00000100:00100000:3.0:1787784629.811040:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657953152 00000100:00100000:3.0:1787784629.811132:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 126 00000100:00100000:3.0:1787784629.811142:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:18622:x1874627657953152:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784629.811640:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:18622:x1874627657953152:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 498us (710us total) trans 4294967704 rc 0/0 00000100:00100000:3.0:1787784629.811649:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 126 00010000:00010000:2.0:1787784630.193401:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784630.769359:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668325888:0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.769426:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784630.769483:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668326400:0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.769521:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784630.769542:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784630.769548:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668326016:0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.769571:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784630.769587:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668326528:0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.769627:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784630.769654:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668326144:0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.769691:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784630.769709:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784630.769714:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668326272:0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.769735:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784630.769818:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668326400 00000100:00100000:2.0:1787784630.769824:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668326400 00000100:00100000:2.0:1787784630.769880:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668326016 00000100:00100000:2.0:1787784630.769884:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668326016 00000100:00100000:2.0:1787784630.769896:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668326144 00000100:00100000:2.0:1787784630.769898:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668326144 00000100:00100000:2.0:1787784630.769925:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1787784630.769935:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668326400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.769986:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668326400:12345-0@lo:400:kworker.0 Request processed in 53us (464us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784630.769993:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1787784630.770014:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:2.0:1787784630.770020:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668326016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.770046:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668326016:12345-0@lo:400:kworker.0 Request processed in 26us (474us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784630.770054:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:0.0:1787784630.770058:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668325888 00000100:00100000:0.0:1787784630.770065:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668325888 00000100:00100000:2.0:1787784630.770068:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:2.0:1787784630.770077:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668326144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784630.770101:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:0.0:1787784630.770111:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668325888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.770117:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668326144:12345-0@lo:400:kworker.0 Request processed in 39us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784630.770124:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:2.0:1787784630.770169:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668326528 00000100:00100000:0.0:1787784630.770173:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668325888:12345-0@lo:400:kworker.0 Request processed in 63us (746us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784630.770174:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668326528 00000100:00100000:0.0:1787784630.770183:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1787784630.770205:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668326272 00000100:00100000:2.0:1787784630.770209:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668326272 00000100:00100000:2.0:1787784630.770234:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:2.0:1787784630.770242:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668326528:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784630.770257:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1787784630.770267:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668326272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784630.770276:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668326528:12345-0@lo:400:kworker.0 Request processed in 34us (647us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784630.770293:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1787784630.770310:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668326272:12345-0@lo:400:kworker.0 Request processed in 44us (572us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784630.770325:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1787784630.771399:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784630.771427:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668326656:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784630.771468:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1787784630.772358:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784630.772365:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784630.772369:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784630.772387:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668326656 00000100:00100000:1.0:1787784630.772683:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668326656 00000100:00100000:1.0:1787784630.772741:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 109 00000100:00100000:1.0:1787784630.772752:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668326656:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784630.772790:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784630.772987:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668326656:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784630.773029:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668326656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 129us (1409us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784630.773039:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 109 00000100:00100000:1.0:1787784630.773220:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668326272:0@lo:400:kworker.0 00000100:00100000:1.0:1787784630.773243:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668326144:0@lo:400:kworker.0 00000100:00100000:1.0:1787784630.773287:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668326016:0@lo:400:kworker.0 00000100:00100000:1.0:1787784630.773301:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668326528:0@lo:400:kworker.0 00000100:00100000:1.0:1787784630.773316:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668326400:0@lo:400:kworker.0 00000100:00100000:1.0:1787784630.773329:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668325888:0@lo:400:kworker.0 00000001:02000400:2.0:1787784631.016947:0:16105:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 18:50:29 (1787784629) 00000040:00080000:3.0:1787784631.034351:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784631.034359:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784631.034416:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 14 count 2 00000040:00080000:3.0:1787784631.034426:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784631.191365:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784631.191447:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657953408 00000100:00100000:0.0:1787784631.191456:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657953408 00000100:00100000:0.0:1787784631.191522:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 587 00000100:00100000:0.0:1787784631.191533:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+25:19283:x1874627657953408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784631.191555:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784631.191573:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc41c3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b063c expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.191640:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.191656:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.191680:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.191696:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.191704:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.191711:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192115:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc41d8 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192126:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc41d8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192132:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc41d8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784631.192141:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:0.0:1787784631.192310:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192319:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192326:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192331:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784631.192335:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192341:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192354:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:0.0:1787784631.192356:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.192360:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192376:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ca lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192387:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192392:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192398:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192403:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784631.192406:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192412:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192418:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff98950a162d00) 00010000:00010000:0.0:1787784631.192421:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.192423:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192428:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41d1 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192793:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc41c3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b063c expref: 28 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192809:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc41d8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b063c expref: 27 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.192822:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784631.192825:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.192827:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd200, rc 0) 00000100:00100000:0.0:1787784631.192901:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19283:x1874627657953408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1369us (1584us total) trans 4294967705 rc 0/0 00000100:00100000:0.0:1787784631.192910:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 587 00000100:00080000:1.0:1787784631.193064:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784631.198794:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784631.198842:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784631.198905:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657953536 00000100:00100000:0.0:1787784631.198915:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657953536 00000100:00100000:0.0:1787784631.198971:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00100000:0.0:1787784631.198980:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19283:x1874627657953536:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1787784631.199097:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19283:x1874627657953536:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 117us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784631.199105:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00100000:3.0:1787784631.199432:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657953664 00000100:00100000:3.0:1787784631.199437:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657953664 00000100:00100000:3.0:1787784631.199482:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 21 00000100:00100000:3.0:1787784631.199490:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:2355:x1874627657953664:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784631.199501:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784631.199526:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc41d8 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b063c expref: 27 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.199542:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784631.199547:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784631.199579:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2355:x1874627657953664:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 90us (736us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784631.199587:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 21 00000100:00100000:2.0:1787784631.209087:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784631.209764:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657953792 00000100:00100000:0.0:1787784631.209775:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657953792 00000100:00100000:0.0:1787784631.209854:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00100000:0.0:1787784631.209865:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:19284:x1874627657953792:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784631.209896:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784631.209917:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41e6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0643 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.209993:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210007:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210030:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210045:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210053:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210059:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210328:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210337:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210342:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210348:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784631.210352:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210359:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210370:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:0.0:1787784631.210374:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.210378:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.210390:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc41ed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784631.211572:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784631.211822:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.211833:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.211840:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.211846:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784631.211851:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.211858:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.211870:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x80c094a7 (ffff98950a162d00) 00010000:00010000:0.0:1787784631.211978:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.211984:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.211990:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc41f4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.212036:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41e6 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0643 expref: 60 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.212064:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:0.0:1787784631.212066:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.212070:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc41e6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0643 expref: 60 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.212079:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc400, rc 0) 00000100:00100000:0.0:1787784631.212394:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:19284:x1874627657953792:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2529us (3314us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784631.212470:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00100000:0.0:1787784631.217029:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784631.217107:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954048 00000100:00100000:2.0:1787784631.217117:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954048 00000100:00100000:2.0:1787784631.217312:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 590 00000100:00100000:2.0:1787784631.217323:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:19284:x1874627657954048:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:2.0:1787784631.217380:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784631.217407:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc3ffc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b059b expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.217426:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:2.0:1787784631.217431:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784631.217882:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.217897:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.217906:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.218570:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 25 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.218603:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 25 pid: 5812 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784631.218618:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668326784:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784631.219352:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643540 going to sleep for 11 seconds 00010000:00010000:3.0:1787784631.221555:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784631.226439:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668326784:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784631.226485:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784631.226774:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784631.227421:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954176 00000100:00100000:2.0:1787784631.227429:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954176 00010000:00010000:3.0:1787784631.227551:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 26 pid: 5812 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784631.227604:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 176 00000100:00100000:3.0:1787784631.227616:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2356:x1874627657954176:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784631.227632:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784631.227638:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 26 pid: 5812 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.227650:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 26 pid: 5812 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.227822:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4065 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05b7 expref: 26 pid: 5812 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.227838:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784631.227843:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.227852:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.227859:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.227891:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784631.227946:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2356:x1874627657954176:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 333us (1173us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784631.227954:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 176 00010000:00010000:2.0:1787784631.228284:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.228293:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.228320:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.228331:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.228340:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784631.229948:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 512 00010000:00010000:3.0:1787784631.229964:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.229974:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.229982:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.229989:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.229994:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.230010:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.230024:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784631.230027:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.230032:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.230038:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc41fb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784631.230057:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008cd8a41a reply state 000000009c7894cc transno 4294967706 ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784631.230065:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784631.230390:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:19284:x1874627657954048:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 13065us (13367us total) trans 4294967706 rc 0/0 00000100:00100000:3.0:1787784631.230400:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 590 00010000:00010000:3.0:1787784631.235404:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.235421:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.235429:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.235435:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.235441:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.235466:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.235482:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x80c094a7 (ffff989519d44d00) 00010000:00010000:3.0:1787784631.235485:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.235489:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.235496:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc4202 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784631.240234:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.240466:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954304 00000100:00100000:3.0:1787784631.240476:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954304 00000100:00100000:3.0:1787784631.240523:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 591 00000100:00100000:3.0:1787784631.240532:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+59:19285:x1874627657954304:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784631.240555:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784631.240577:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4209 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b064a expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.240641:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4210 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.240656:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4210 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.240685:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4210 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784631.240743:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784631.240761:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 15 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784631.240791:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4210 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.240817:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4209 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b064a expref: 61 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.240836:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4210 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b064a expref: 60 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.240845:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784631.240849:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784631.240852:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905000, rc 0) 00000100:00100000:3.0:1787784631.240912:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:19285:x1874627657954304:12345-192.168.203.43@tcp:101:bash.0 Request processed in 380us (687us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784631.240919:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 591 00000100:00100000:0.0:1787784631.244183:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784631.244616:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954432 00000100:00100000:1.0:1787784631.244624:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954432 00000100:00100000:1.0:1787784631.244687:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 592 00000100:00100000:1.0:1787784631.244698:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+60:19285:x1874627657954432:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784631.244718:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784631.244738:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ade00/0xb0ce63137acc4217 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0651 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.244802:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.244816:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.244843:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.244858:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.244863:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.244867:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245032:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010acc00/0xb0ce63137acc422c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245041:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010acc00/0xb0ce63137acc422c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245048:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010acc00/0xb0ce63137acc422c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784631.245080:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 512 00000004:00200000:1.0:1787784631.245083:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784631.245101:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894010acc00/0xb0ce63137acc422c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245110:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245115:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245126:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245130:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784631.245133:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245138:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245184:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:1.0:1787784631.245187:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784631.245192:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245199:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc421e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245212:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245218:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245224:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245229:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784631.245232:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245247:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245253:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x80c094a7 (ffff9895029d1d00) 00010000:00010000:1.0:1787784631.245255:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784631.245257:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245261:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4225 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245286:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ade00/0xb0ce63137acc4217 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0651 expref: 62 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245301:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010acc00/0xb0ce63137acc422c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0651 expref: 61 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.245308:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:1.0:1787784631.245310:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784631.245312:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010acc00, rc 0) 00000100:00100000:1.0:1787784631.245388:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+61:19285:x1874627657954432:12345-192.168.203.43@tcp:101:bash.0 Request processed in 691us (1242us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784631.245622:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 592 00000100:00100000:0.0:1787784631.254073:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784631.254484:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954560 00000100:00100000:2.0:1787784631.254494:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954560 00000100:00100000:3.0:1787784631.254775:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 593 00000100:00100000:3.0:1787784631.254786:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+61:19285:x1874627657954560:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784631.254859:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784631.254879:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4233 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0658 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.254946:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.254960:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.254971:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.254998:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255006:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255013:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255083:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255091:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255098:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255104:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.255109:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255116:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255128:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:3.0:1787784631.255131:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.255135:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255141:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc423a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255333:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255341:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255347:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255352:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.255355:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255362:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255371:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff989505542f00) 00010000:00010000:3.0:1787784631.255374:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.255377:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255383:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4241 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255420:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4233 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0658 expref: 62 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255431:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:3.0:1787784631.255432:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.255442:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4233 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0658 expref: 62 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.255451:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905200, rc 0) 00000100:00100000:3.0:1787784631.255511:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+61:19285:x1874627657954560:12345-192.168.203.43@tcp:101:bash.0 Request processed in 726us (1447us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784631.255518:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 593 00000100:00100000:2.0:1787784631.266803:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.268106:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954688 00000100:00100000:3.0:1787784631.268126:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954688 00000100:00100000:3.0:1787784631.268265:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 594 00000100:00100000:3.0:1787784631.268287:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:19286:x1874627657954688:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784631.268315:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784631.268345:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4248 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b065f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.268433:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.268451:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.268465:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784631.268543:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784631.268549:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 15 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784631.268580:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.268611:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4248 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 27 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.268654:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 26 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.268739:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784631.268743:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784631.268747:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655a00, rc 0) 00000100:00100000:3.0:1787784631.268829:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:19286:x1874627657954688:12345-192.168.203.43@tcp:101:rm.0 Request processed in 544us (2033us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784631.268837:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 594 00000100:00100000:0.0:1787784631.272318:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.272413:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954816 00000100:00100000:3.0:1787784631.272427:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954816 00000100:00100000:3.0:1787784631.272500:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 595 00000100:00100000:3.0:1787784631.272513:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+26:19286:x1874627657954816:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784631.272541:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784631.272573:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654800/0xb0ce63137acc4256 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0666 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272656:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272682:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272695:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272713:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272741:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272749:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272897:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc426b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272907:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc426b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.272915:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc426b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784631.272955:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 512 00000004:00200000:3.0:1787784631.272959:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784631.272988:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc426b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273004:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273010:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273018:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273024:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.273029:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273043:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273057:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:3.0:1787784631.273060:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.273065:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273072:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655200/0xb0ce63137acc425d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273090:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273096:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273102:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273107:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989540655800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.273110:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273117:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273124:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x80c094a7 (ffff989505542f00) 00010000:00010000:3.0:1787784631.273127:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.273130:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273199:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655800/0xb0ce63137acc4264 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273239:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654800/0xb0ce63137acc4256 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0666 expref: 28 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273253:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540654000/0xb0ce63137acc426b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0666 expref: 27 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.273261:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:3.0:1787784631.273264:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784631.273266:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540654000, rc 0) 00000100:00100000:3.0:1787784631.273350:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19286:x1874627657954816:12345-192.168.203.43@tcp:101:rm.0 Request processed in 840us (1045us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784631.273359:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 595 00000100:00100000:0.0:1787784631.280994:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.281333:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657954944 00000100:00100000:3.0:1787784631.281343:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657954944 00000100:00100000:3.0:1787784631.281488:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 596 00000100:00100000:3.0:1787784631.281499:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19286:x1874627657954944:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784631.281518:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784631.281538:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4272 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b066d expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.281602:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.281617:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.281649:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.281682:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.281692:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.281701:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.281897:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.281913:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.281934:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.281941:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784631.281945:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.281954:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.281972:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:2.0:1787784631.281975:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784631.281980:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.281987:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4279 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282003:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282010:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282016:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282021:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784631.282024:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282041:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282050:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff989505542800) 00010000:00010000:2.0:1787784631.282052:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784631.282056:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282063:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4280 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282121:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4272 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b066d expref: 28 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282133:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:2.0:1787784631.282136:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784631.282141:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655c00/0xb0ce63137acc4272 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b066d expref: 28 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.282202:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989540655c00, rc 0) 00000100:00100000:2.0:1787784631.282291:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19286:x1874627657954944:12345-192.168.203.43@tcp:101:rm.0 Request processed in 792us (1305us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784631.282301:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 596 00000100:00100000:2.0:1787784631.297473:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.297602:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657955072 00000100:00100000:3.0:1787784631.297616:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657955072 00000100:00100000:3.0:1787784631.297700:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 597 00000100:00100000:3.0:1787784631.297710:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19286:x1874627657955072:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784631.297729:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784631.297749:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4287 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0674 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297817:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297831:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297842:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297857:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297879:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297886:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297959:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297968:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297974:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297980:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.297985:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.297992:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298004:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:3.0:1787784631.298008:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.298013:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298019:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc428e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298034:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298046:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298052:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298057:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.298060:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298067:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298075:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff989505542f00) 00010000:00010000:3.0:1787784631.298078:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.298081:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298087:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4295 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298120:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4287 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0674 expref: 28 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298295:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:3.0:1787784631.298298:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.298303:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4287 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0674 expref: 28 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.298326:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784631.298389:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19286:x1874627657955072:12345-192.168.203.43@tcp:101:rm.0 Request processed in 681us (931us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784631.298398:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 597 00000100:00100000:0.0:1787784631.320647:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784631.324901:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657955200 00000100:00100000:1.0:1787784631.324925:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657955200 00000100:00100000:1.0:1787784631.325031:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 598 00000100:00100000:1.0:1787784631.325044:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19287:x1874627657955200:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784631.325096:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784631.325127:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ade00/0xb0ce63137acc429c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b067b expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325238:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325258:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325272:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325288:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325295:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325649:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325825:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325834:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325842:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325849:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784631.325863:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325873:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325888:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:1.0:1787784631.325891:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784631.325896:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325903:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc42a3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325920:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325927:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325933:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325938:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784631.325942:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325948:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325955:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff9895029d1d00) 00010000:00010000:1.0:1787784631.325958:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784631.325967:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.325973:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc42aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.326017:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ade00/0xb0ce63137acc429c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b067b expref: 28 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.326029:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:1.0:1787784631.326031:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784631.326035:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ade00/0xb0ce63137acc429c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b067b expref: 28 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.326045:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ade00, rc 0) 00000100:00100000:1.0:1787784631.326238:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19287:x1874627657955200:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1196us (5607us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784631.326249:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 598 00000100:00100000:0.0:1787784631.336519:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.337130:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657955328 00000100:00100000:3.0:1787784631.337141:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657955328 00000100:00100000:3.0:1787784631.337345:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:3.0:1787784631.337356:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19287:x1874627657955328:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784631.337376:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784631.337398:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc42b1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0682 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337465:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337560:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337574:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337591:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337600:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337607:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337759:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337768:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337774:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337780:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.337785:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337792:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.337804:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:3.0:1787784631.337807:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.337810:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338117:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc42b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338137:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338143:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338363:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338379:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784631.338383:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338389:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338399:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff989505542f00) 00010000:00010000:3.0:1787784631.338402:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.338405:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338412:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc42bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338455:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc42b1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0682 expref: 28 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338466:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:3.0:1787784631.338469:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784631.338485:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc42b1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0682 expref: 28 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.338495:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784631.338568:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19287:x1874627657955328:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1213us (2058us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784631.338577:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:2.0:1787784631.345427:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784631.345526:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657955584 00000100:00100000:0.0:1787784631.345536:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657955584 00000100:00100000:0.0:1787784631.345596:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 600 00000100:00100000:0.0:1787784631.345606:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19287:x1874627657955584:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784631.345642:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784631.345683:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc42c6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0689 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.345815:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.345829:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.345841:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.345856:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.345863:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.345870:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784631.346865:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xe:0x0], index 1: 0 00010000:00010000:0.0:1787784631.349033:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd800/0xb0ce63137acc42db lrc: 3/1,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349054:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd800/0xb0ce63137acc42db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349066:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd800/0xb0ce63137acc42db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349118:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349125:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349132:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349138:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784631.349142:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349176:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349188:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:0.0:1787784631.349191:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.349198:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349214:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc42cd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784631.349229:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003cbac395 reply state 00000000d4416518 transno 4294967707 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784631.349237:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784631.349281:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc42c6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0689 expref: 30 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349309:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd800/0xb0ce63137acc42db lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0689 expref: 29 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.349316:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff98950a162c00) 00010000:00010000:0.0:1787784631.349319:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784631.349321:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd800, rc 0) 00000100:00100000:0.0:1787784631.349400:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:19287:x1874627657955584:12345-192.168.203.43@tcp:101:cp.0 Request processed in 3795us (3981us total) trans 4294967707 rc 0/0 00000100:00100000:0.0:1787784631.349410:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 600 00010000:00010000:0.0:1787784631.351815:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.351831:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.351838:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.351844:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784631.351849:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.351857:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.351872:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff98950a162d00) 00010000:00010000:0.0:1787784631.351875:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784631.351892:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.351913:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc42d4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784631.353040:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784631.353217:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657955712 00000100:00100000:0.0:1787784631.353227:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657955712 00000100:00100000:0.0:1787784631.353287:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 153 00000100:00100000:0.0:1787784631.353298:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657955712:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784631.353315:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784631.353335:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0690 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784631.353359:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784631.353377:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=422s v=(0 0) 00010000:00010000:0.0:1787784631.353382:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784631.353390:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 8 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.353406:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 8 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784631.353419:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbde00, rc 0) 00000100:00100000:0.0:1787784631.353467:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657955712:12345-192.168.203.43@tcp:101:cp.0 Request processed in 170us (444us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784631.353474:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 153 00000100:00080000:1.0:1787784631.353604:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784631.359654:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784631.359760:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657955840 00000100:00100000:1.0:1787784631.359770:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657955840 00000100:00100000:1.0:1787784631.359830:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 154 00000100:00100000:1.0:1787784631.359840:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:19287:x1874627657955840:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784631.359857:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784631.359874:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0697 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.359918:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.359947:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784631.359951:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784631.359957:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.359991:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.360004:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 9 pid: 6670 timeout: 522 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784631.360029:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668326912:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784631.360117:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950133a600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784631.365916:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.365994:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657955968 00000100:00100000:3.0:1787784631.366016:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657955968 00010000:00010000:3.0:1787784631.366063:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 10 pid: 6670 timeout: 522 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784631.366104:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:3.0:1787784631.366114:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657955968:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784631.366136:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784631.366204:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 10 pid: 6670 timeout: 522 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.366220:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 10 pid: 6670 timeout: 522 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.366244:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784631.366248:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784631.366267:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.366277:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784631.366313:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784631.366326:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668327040:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784631.366390:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce180 going to sleep for 11 seconds 00010000:00010000:2.0:1787784631.367103:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000476f99e8/0xb0ce63137acc42e9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0690 expref: 10 pid: 6670 timeout: 522 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784631.367405:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668326912:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784631.367431:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.367443:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac800, rc 0) 00000100:00100000:2.0:1787784631.367489:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:19287:x1874627657955840:12345-192.168.203.43@tcp:101:cp.0 Request processed in 7651us (7844us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784631.367497:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 154 00000100:00100000:3.0:1787784631.370765:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668327040:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784631.370801:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784631.371459:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627657955968:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5339us (5543us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784631.371477:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:0.0:1787784631.376554:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.376689:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657956224 00000100:00100000:3.0:1787784631.376701:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657956224 00000100:00100000:3.0:1787784631.376776:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 127 00000100:00100000:3.0:1787784631.376789:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19287:x1874627657956224:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:3.0:1787784631.378084:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:19287:x1874627657956224:12345-192.168.203.43@tcp:35:cp.0 Request processed in 1288us (1534us total) trans 4294967708 rc 0/0 00000100:00100000:3.0:1787784631.378097:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 127 00000100:00100000:1.0:1787784631.410347:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784631.410546:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657956352 00000100:00100000:2.0:1787784631.410557:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657956352 00000100:00100000:2.0:1787784631.410688:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 601 00000100:00100000:2.0:1787784631.410701:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+61:19290:x1874627657956352:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784631.410728:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784631.410762:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc42f7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b069e expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.410942:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.410961:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.410975:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.410993:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411019:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411027:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411291:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc430c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411301:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc430c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411309:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc430c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784631.411351:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xf:0x0]: returning size 158 00000004:00200000:2.0:1787784631.411356:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784631.411378:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc430c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411392:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc430c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411402:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411462:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411472:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411493:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784631.411499:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411507:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411523:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:2.0:1787784631.411526:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784631.411532:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411538:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc42fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411554:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411560:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411566:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411571:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784631.411574:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411580:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411597:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff989519d44600) 00010000:00010000:2.0:1787784631.411600:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784631.411603:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411609:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4305 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411649:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc42f7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b069e expref: 63 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411664:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098600/0xb0ce63137acc430c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b069e expref: 62 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.411721:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff98950a162c00) 00010000:00010000:2.0:1787784631.411724:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784631.411727:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098600, rc 0) 00000100:00100000:2.0:1787784631.411918:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:19290:x1874627657956352:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1219us (1583us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784631.411927:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 601 00000100:00100000:1.0:1787784631.416630:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784631.417108:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657956480 00000100:00100000:0.0:1787784631.417138:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657956480 00000100:00100000:0.0:1787784631.417384:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 155 00000100:00100000:0.0:1787784631.417396:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2354:x1874627657956480:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784631.417420:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784631.417445:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003d5400fa/0xb0ce63137acc4313 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06a5 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784631.417527:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895e940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668327168:192.168.203.43@tcp:106: 00000100:00080000:1.0:1787784631.417576:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784631.417640:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504c5b240 going to sleep for 11 seconds 00002000:00010000:1.0:1787784631.420033:0:10688:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size: 0 -> 158 00002000:00010000:1.0:1787784631.420043:0:10688:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime: 0 -> 1787784630 00002000:00010000:1.0:1787784631.420045:0:10688:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb atime: 0 -> 1787784630 00002000:00010000:1.0:1787784631.420047:0:10688:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime: 0 -> 1787784630 00000100:00100000:1.0:1787784631.420073:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895e940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668327168:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784631.420112:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003d5400fa/0xb0ce63137acc4313 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06a5 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.420205:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003d5400fa/0xb0ce63137acc4313 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06a5 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784631.420222:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd600, rc 0) 00000100:00100000:1.0:1787784631.420300:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2354:x1874627657956480:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2906us (3685us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784631.420311:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 155 00000100:00100000:3.0:1787784631.423164:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784631.423399:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657956608 00000100:00100000:2.0:1787784631.423415:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657956608 00000100:00100000:2.0:1787784631.423534:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 602 00000100:00100000:2.0:1787784631.423549:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:19290:x1874627657956608:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784631.423581:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784631.423612:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc431a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06ac expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.423974:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4321 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.423993:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4321 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.424006:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4321 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784631.424085:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 512 00000004:00200000:2.0:1787784631.424090:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784631.424118:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4321 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.424175:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc431a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06ac expref: 64 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.424198:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099800/0xb0ce63137acc4321 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06ac expref: 63 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.424206:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff9895029d1300) 00010000:00010000:2.0:1787784631.424209:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784631.424213:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401099800, rc 0) 00000100:00100000:2.0:1787784631.424395:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19290:x1874627657956608:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 849us (1262us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784631.424404:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 602 00000100:00100000:3.0:1787784631.427496:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784631.427620:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657956736 00000100:00100000:2.0:1787784631.427633:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657956736 00000100:00100000:2.0:1787784631.427971:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 156 00000100:00100000:2.0:1787784631.427984:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2355:x1874627657956736:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784631.428009:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784631.428035:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e43d2ecf/0xb0ce63137acc4328 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06b3 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784631.428114:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668327296:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784631.428854:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643300 going to sleep for 11 seconds 00000100:00100000:2.0:1787784631.430685:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668327296:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784631.430744:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e43d2ecf/0xb0ce63137acc4328 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06b3 expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.430842:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e43d2ecf/0xb0ce63137acc4328 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06b3 expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784631.430862:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098e00, rc 0) 00000100:00100000:2.0:1787784631.430993:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2355:x1874627657956736:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3010us (3510us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784631.431003:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 156 00000100:00100000:1.0:1787784631.436256:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784631.436520:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657956992 00000100:00100000:3.0:1787784631.436533:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657956992 00000100:00100000:3.0:1787784631.436973:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 603 00000100:00100000:3.0:1787784631.436986:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19290:x1874627657956992:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784631.437400:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].14, 136 off10096 00000100:00100000:3.0:1787784631.437529:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19290:x1874627657956992:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 541us (1284us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784631.437538:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 603 00000040:00080000:2.0:1787784632.055552:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784632.055565:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784632.055611:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 3 00000040:00080000:2.0:1787784632.055620:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784632.055964:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668327424:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784632.056092:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784632.063360:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668327424:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784632.067043:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784632.067484:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657957120 00000100:00100000:3.0:1787784632.067498:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657957120 00000100:00100000:3.0:1787784632.067589:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 604 00000100:00100000:3.0:1787784632.067602:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:19300:x1874627657957120:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784632.067672:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xf:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784632.067816:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:19300:x1874627657957120:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 216us (785us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784632.067825:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 604 00000100:00100000:1.0:1787784632.071762:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784632.072026:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657957376 00000100:00100000:3.0:1787784632.072040:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657957376 00000100:00100000:3.0:1787784632.072108:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 605 00000100:00100000:3.0:1787784632.072119:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:19300:x1874627657957376:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784632.072175:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784632.072291:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc432f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06ba expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784632.072431:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4336 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784632.072449:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4336 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784632.072466:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4336 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784632.072478:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xf:0x0]: returning size 158 00010000:00010000:3.0:1787784632.072633:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc432f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06ba expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784632.072657:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4336 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06ba expref: 32 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784632.072696:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff98950a162c00) 00010000:00010000:3.0:1787784632.072700:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784632.072703:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3400, rc 0) 00000100:00100000:3.0:1787784632.072833:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19300:x1874627657957376:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 714us (1085us total) trans 4294967709 rc 0/0 00000100:00100000:3.0:1787784632.072841:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 605 00000100:00100000:1.0:1787784632.079088:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784632.079179:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784632.079184:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657957504 00000100:00100000:2.0:1787784632.079199:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657957504 00000100:00100000:2.0:1787784632.079284:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 157 00000100:00100000:2.0:1787784632.079298:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657957504:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784632.079357:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.079381:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 8 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.079391:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784632.079396:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784632.079401:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 4/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.079444:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 8 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.079460:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 8 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784632.079477:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668327552:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784632.079581:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4000 going to sleep for 11 seconds 00000100:00100000:3.0:1787784632.079625:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657957632 00000100:00100000:3.0:1787784632.079631:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657957632 00000100:00100000:3.0:1787784632.079847:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 22 00000100:00100000:3.0:1787784632.079857:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+32:2356:x1874627657957632:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784632.079870:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784632.079896:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4336 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06ba expref: 32 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784632.079913:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff98950a162c00) 00010000:00010000:3.0:1787784632.079918:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784632.079949:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2356:x1874627657957632:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 95us (771us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784632.079957:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 22 00000100:00100000:3.0:1787784632.082085:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668327552:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784632.082135:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784632.084180:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784632.084294:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657957760 00000100:00100000:3.0:1787784632.084314:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657957760 00002000:00100000:3.0:1787784632.084380:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xe:0x0] (0->157) req@ffff989402b11a40 x1874627657957760/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:513/0 lens 648/0 e 0 to 0 dl 1787784643 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784632.084392:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402b11a40 with x1874627657957760 ext(0->157) 00010000:00010000:3.0:1787784632.084402:0:6676:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 9 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784632.084421:0:6676:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 9 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784632.084430:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989402b11a40 00000100:00100000:3.0:1787784632.084472:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 4 00000100:00100000:3.0:1787784632.084484:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657957760:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784632.085582:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657957760:12345-192.168.203.43@tcp:4:cp.0 Request processed in 1091us (1435us total) trans 4294967329 rc 0/0 00000100:00100000:2.0:1787784632.085595:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 4 00002000:00100000:2.0:1787784632.085614:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xe:0x0] (0->157) req@ffff989402b11a40 x1874627657957760/t4294967329(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:513/0 lens 648/448 e 0 to 0 dl 1787784643 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784632.085623:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402b11a40 with x1874627657957760 ext(0->157) 00010000:00010000:2.0:1787784632.085631:0:6676:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 11 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.085646:0:6676:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 11 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784632.085653:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989402b11a40 00000100:00100000:2.0:1787784632.088788:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784632.088845:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657957888 00000100:00100000:0.0:1787784632.088858:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657957888 00010000:00010000:0.0:1787784632.088907:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 11 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784632.088947:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:0.0:1787784632.088956:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657957888:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784632.088970:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784632.088992:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 11 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.089004:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 11 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.089029:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000020b9b99d/0xb0ce63137acc42f0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0697 expref: 11 pid: 13825 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.089055:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784632.089058:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784632.089073:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.089082:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.089111:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784632.089208:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657957888:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 252us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784632.089216:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 178 00010000:00010000:2.0:1787784632.089229:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089238:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089268:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089278:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089285:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089443:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784632.089448:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089459:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089475:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.089483:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b20da214/0xb0ce63137acc4344 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784632.089547:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657957504:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 10254us (10472us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784632.089556:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 157 00000100:00100000:1.0:1787784632.093815:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784632.094580:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657958144 00000100:00100000:3.0:1787784632.094595:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657958144 00000100:00100000:3.0:1787784632.094685:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 606 00000100:00100000:3.0:1787784632.094698:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:19300:x1874627657958144:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784632.094850:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:19300:x1874627657958144:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 153us (1049us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784632.094858:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 606 00000100:00100000:3.0:1787784632.101915:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784632.102541:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657958272 00000100:00100000:1.0:1787784632.102559:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657958272 00000100:00100000:1.0:1787784632.102646:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 158 00000100:00100000:1.0:1787784632.102675:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657958272:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784632.102993:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784632.103030:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc434b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06c1 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784632.103080:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 158, mtime: 0x6a8f6db6, atime: 0x0, ctime: 0x6a8f6db6, blocks: 0x1 00010000:00010000:1.0:1787784632.103119:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=423s v=(0 0) 00010000:00010000:1.0:1787784632.103128:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784632.103171:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc434b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c1 expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.103197:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077264259/0xb0ce63137acc434b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c1 expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.103214:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ade00, rc 0) 00000100:00100000:1.0:1787784632.103310:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657958272:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 653us (1591us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784632.103319:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 158 00000100:00100000:3.0:1787784632.106740:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784632.106813:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657958528 00000100:00100000:1.0:1787784632.106826:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657958528 00000100:00100000:1.0:1787784632.106903:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 607 00000100:00100000:1.0:1787784632.106916:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:19300:x1874627657958528:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784632.107088:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:19300:x1874627657958528:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 174us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784632.107097:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 607 00000100:00100000:1.0:1787784632.121142:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784632.121577:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657958656 00000100:00100000:3.0:1787784632.121592:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657958656 00002000:00100000:3.0:1787784632.121676:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xe:0x0] (0->4095) req@ffff98954028b0c0 x1874627657958656/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:513/0 lens 488/0 e 0 to 0 dl 1787784643 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784632.121711:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954028b0c0 with x1874627657958656 ext(0->4095) 00002000:00010000:3.0:1787784632.121729:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98954028b0c0 00000100:00100000:3.0:1787784632.121784:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 54 00000100:00100000:3.0:1787784632.121799:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657958656:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784632.122084:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657958656:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 288us (955us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784632.122093:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 54 00002000:00100000:3.0:1787784632.122102:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xe:0x0] (0->4095) req@ffff98954028b0c0 x1874627657958656/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:513/0 lens 488/4536 e 0 to 0 dl 1787784643 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784632.122111:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954028b0c0 with x1874627657958656 ext(0->4095) 00002000:00010000:3.0:1787784632.122114:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98954028b0c0 00000100:00100000:3.0:1787784632.133715:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784632.133898:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657958784 00000100:00100000:1.0:1787784632.133913:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657958784 00000100:00100000:1.0:1787784632.133991:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 608 00000100:00100000:1.0:1787784632.134002:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+31:19300:x1874627657958784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784632.134030:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784632.134080:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4352 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06c8 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.134179:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.134197:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.134213:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.135026:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc4352 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 33 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.135053:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 32 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.135064:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff98950a162c00) 00010000:00010000:1.0:1787784632.135068:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784632.135073:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad000, rc 0) 00000100:00100000:1.0:1787784632.135195:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19300:x1874627657958784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1193us (1495us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784632.135205:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 608 00000100:00100000:0.0:1787784632.141906:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784632.142183:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657958912 00000100:00100000:2.0:1787784632.142196:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657958912 00000100:00100000:2.0:1787784632.142384:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 609 00000100:00100000:2.0:1787784632.142397:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19300:x1874627657958912:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784632.142441:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xf:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784632.143078:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19300:x1874627657958912:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 680us (1182us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784632.143089:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 609 00000100:00100000:0.0:1787784632.147112:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784632.147790:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657959040 00000100:00100000:2.0:1787784632.147805:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657959040 00000100:00100000:2.0:1787784632.147875:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 159 00000100:00100000:2.0:1787784632.147886:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657959040:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784632.147944:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.147966:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784632.147970:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784632.147977:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.147994:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.148039:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.148048:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.148058:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.148066:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784632.148071:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.148093:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.148111:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.148119:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000739a929d/0xb0ce63137acc4360 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784632.148320:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657959040:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 434us (1220us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784632.148330:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 159 00000100:00100000:2.0:1787784632.151374:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784632.151449:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657959296 00000100:00100000:0.0:1787784632.151462:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657959296 00000100:00100000:0.0:1787784632.151536:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 610 00000100:00100000:0.0:1787784632.151549:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19300:x1874627657959296:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784632.151654:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.151693:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 32 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.151704:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.151766:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 32 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784632.151783:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 32 pid: 5812 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784632.151809:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668327680:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784632.151929:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506773f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784632.158141:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668327680:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784632.158226:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784632.158914:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784632.159954:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657959424 00000100:00100000:2.0:1787784632.159968:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657959424 00010000:00010000:2.0:1787784632.160025:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 33 pid: 5812 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784632.160505:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 179 00000100:00100000:2.0:1787784632.160520:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+33:2357:x1874627657959424:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784632.160543:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784632.160553:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 33 pid: 5812 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.160569:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 33 pid: 5812 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.160607:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4359 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06c8 expref: 33 pid: 5812 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.160625:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff98950a162c00) 00010000:00010000:2.0:1787784632.160632:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.160642:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.160651:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784632.160708:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784632.160788:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+32:2357:x1874627657959424:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 270us (1876us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784632.160798:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 179 00010000:00010000:1.0:1787784632.160906:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.160922:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.161181:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.161195:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.161203:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.161210:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784632.161215:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.161223:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.161233:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff98950a162c00) 00010000:00010000:1.0:1787784632.161236:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784632.161240:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.161247:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc4367 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784632.162895:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19300:x1874627657959296:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 11335us (11520us total) trans 4294967710 rc 0/0 00000100:00100000:1.0:1787784632.162909:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 610 00000040:00080000:0.0:1787784632.162971:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784632.162978:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784632.165515:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784632.165537:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784632.172539:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784632.175656:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657959680 00000100:00100000:0.0:1787784632.175685:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657959680 00000100:00100000:0.0:1787784632.175772:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 128 00000100:00100000:0.0:1787784632.175784:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19300:x1874627657959680:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784632.176201:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+33:19300:x1874627657959680:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 418us (3675us total) trans 4294967711 rc 0/0 00000100:00100000:0.0:1787784632.176212:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 128 00000040:00080000:0.0:1787784632.216230:0:16154:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784632.216307:0:16154:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784632.216364:0:16154:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 3 00000040:00080000:0.0:1787784632.216370:0:16154:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784632.241567:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784632.279422:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784632.279544:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657959808 00000100:00100000:1.0:1787784632.279558:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657959808 00000100:00100000:1.0:1787784632.279638:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 160 00000100:00100000:1.0:1787784632.279650:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+8:2354:x1874627657959808:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784632.279705:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784632.279730:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e597a957/0xb0ce63137acc436e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06cf expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.279870:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784632.279877:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784632.279888:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e597a957/0xb0ce63137acc436e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06cf expref: 9 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.279912:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e597a957/0xb0ce63137acc436e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06cf expref: 9 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784632.279930:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ace00, rc 0) 00000100:00100000:1.0:1787784632.280223:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657959808:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 573us (814us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784632.280231:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 160 00000100:00100000:1.0:1787784632.290655:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784632.291188:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960064 00000100:00100000:3.0:1787784632.291212:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960064 00000100:00100000:3.0:1787784632.291288:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 611 00000100:00100000:3.0:1787784632.291301:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19301:x1874627657960064:12345-192.168.203.43@tcp:58:lfs.0 00000100:00100000:3.0:1787784632.291469:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19301:x1874627657960064:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 172us (827us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784632.291488:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 611 00000001:00080000:1.0:1787784632.383004:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967711 is committed 00000001:00080000:1.0:1787784632.383031:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967706 is committed 00000001:00080000:1.0:1787784632.383207:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967329 is committed 00000040:00080000:1.0:1787784633.049046:0:16207:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784633.049060:0:16207:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784633.049116:0:16207:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 3 00000040:00080000:1.0:1787784633.049122:0:16207:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:0.0:1787784633.201489:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784633.201503:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784633.203027:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784633.203046:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784633.266978:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784633.843606:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b535a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668327808:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784633.843730:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784633.844192:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668327808 00000100:00100000:0.0:1787784633.844203:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668327808 00000100:00100000:0.0:1787784633.844292:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 110 00000100:00100000:0.0:1787784633.844304:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668327808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784633.844350:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784633.844433:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668327808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 129us (702us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784633.844442:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 110 00000100:00100000:0.0:1787784633.844516:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b535a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668327808:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1787784633.845030:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784634.124037:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784634.124094:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960320 00000100:00100000:2.0:1787784634.124104:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960320 00000100:00100000:2.0:1787784634.124192:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 249 00000100:00100000:2.0:1787784634.124204:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:2354:x1874627657960320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784634.124277:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:2354:x1874627657960320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 76us (248us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784634.124495:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:3.0:1787784634.124601:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784634.124643:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 249 00000100:00100000:0.0:1787784634.124684:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784634.124717:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784634.124960:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960576 00000100:00100000:2.0:1787784634.124966:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960576 00000100:00100000:0.0:1787784634.124977:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784634.125013:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784634.125037:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784634.125058:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960832 00000100:00100000:0.0:1787784634.125061:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960832 00000100:00100000:2.0:1787784634.125078:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960448 00000100:00100000:2.0:1787784634.125082:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960448 00000100:00100000:0.0:1787784634.125092:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960960 00000100:00100000:0.0:1787784634.125094:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960960 00000100:00100000:2.0:1787784634.125100:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:2.0:1787784634.125106:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657960576:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784634.125107:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 188 00000100:00100000:0.0:1787784634.125112:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657960832:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784634.125164:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657960832:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 51us (480us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784634.125171:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 188 00000100:00100000:3.0:1787784634.125179:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960192 00000100:00100000:3.0:1787784634.125183:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960192 00000100:00100000:3.0:1787784634.125210:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:3.0:1787784634.125216:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657960192:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784634.125278:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657960576:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 170us (783us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784634.125285:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:2.0:1787784634.125298:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:2.0:1787784634.125304:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657960448:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784634.125331:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657960448:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784634.125336:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:2.0:1787784634.125347:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:2.0:1787784634.125354:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657960960:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784634.125689:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657960704 00000100:00100000:0.0:1787784634.125694:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657960704 00000100:00100000:0.0:1787784634.125721:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 250 00000100:00100000:0.0:1787784634.125726:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2354:x1874627657960704:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784634.125776:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657960960:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 423us (738us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784634.125776:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2354:x1874627657960704:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (762us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784634.125783:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 250 00000100:00100000:2.0:1787784634.125784:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00080000:2.0:1787784634.125800:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784634.125803:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784634.125805:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784634.125851:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657960192:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 633us (1132us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784634.125858:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 337 00010000:00010000:2.0:1787784634.289363:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784635.317989:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784635.889495:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668328064:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.889596:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784635.889668:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668328576:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.889701:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784635.889738:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787784635.889745:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668327936:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.889778:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784635.889801:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668328448:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.889824:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784635.889851:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668328192:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.889877:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784635.889896:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784635.889902:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668328320:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.889925:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784635.890487:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668328064 00000100:00100000:2.0:1787784635.890505:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668328064 00000100:00100000:2.0:1787784635.890560:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668328448 00000100:00100000:2.0:1787784635.890564:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668328448 00000100:00100000:2.0:1787784635.890588:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668328192 00000100:00100000:2.0:1787784635.890592:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668328192 00000100:00100000:2.0:1787784635.890612:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:2.0:1787784635.890621:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668328064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784635.890719:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668328064:12345-0@lo:400:kworker.0 Request processed in 99us (1124us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784635.890727:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:2.0:1787784635.890746:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:2.0:1787784635.890752:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668328448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784635.890782:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668328448:12345-0@lo:400:kworker.0 Request processed in 30us (957us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784635.890788:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:2.0:1787784635.890799:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:2.0:1787784635.890806:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668328192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784635.890833:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668328192:12345-0@lo:400:kworker.0 Request processed in 28us (956us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784635.890838:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00080000:2.0:1787784635.890857:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784635.891780:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668328192:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.891817:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668328704:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784635.891865:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784635.891962:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668328576 00000100:00100000:0.0:1787784635.891969:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668328576 00000100:00100000:0.0:1787784635.892022:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668328320 00000100:00100000:0.0:1787784635.892025:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668328320 00000100:00100000:0.0:1787784635.892049:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:0.0:1787784635.892056:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668328576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.892188:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668328576:12345-0@lo:400:kworker.0 Request processed in 132us (2485us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784635.892196:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:0.0:1787784635.892222:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:0.0:1787784635.892228:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668328320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.892263:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668328320:12345-0@lo:400:kworker.0 Request processed in 35us (2336us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784635.892269:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:0.0:1787784635.892290:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668327936 00000100:00100000:0.0:1787784635.892295:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668327936 00000100:00100000:0.0:1787784635.892324:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:0.0:1787784635.892330:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668327936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.892362:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668327936:12345-0@lo:400:kworker.0 Request processed in 31us (2584us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784635.892367:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00080000:0.0:1787784635.892420:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784635.892424:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784635.892446:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668327936:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.892462:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668328448:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.892477:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668328064:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.892499:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668328576:0@lo:400:kworker.0 00000100:00100000:0.0:1787784635.892540:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668328320:0@lo:400:kworker.0 00000100:00100000:1.0:1787784635.893459:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668328704 00000100:00100000:1.0:1787784635.893468:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668328704 00000100:00100000:1.0:1787784635.893512:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:1.0:1787784635.893528:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668328704:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784635.893568:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784635.893628:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668328704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (1763us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784635.893646:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:0.0:1787784635.893751:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668328704:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1787784636.337518:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784637.366551:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784637.560661:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784637.560794:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657961088 00000100:00100000:3.0:1787784637.560806:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657961088 00000100:00100000:3.0:1787784637.560892:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 612 00000100:00100000:3.0:1787784637.560904:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:19283:x1874627657961088:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00080000:0.0:1787784637.560948:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784637.561028:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:19283:x1874627657961088:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 126us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784637.561037:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 612 00000100:00100000:3.0:1787784637.564751:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784637.564804:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657961344 00000100:00100000:3.0:1787784637.564817:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657961344 00000100:00100000:3.0:1787784637.564890:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 129 00000100:00100000:3.0:1787784637.564904:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:19283:x1874627657961344:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784637.565574:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19283:x1874627657961344:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 670us (832us total) trans 4294967712 rc 0/0 00000100:00100000:3.0:1787784637.565593:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 129 00010000:00010000:2.0:1787784638.385429:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784638.964361:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784638.964397:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668328832:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784638.964515:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784638.964606:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668328832 00000100:00100000:2.0:1787784638.964626:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668328832 00000100:00100000:2.0:1787784638.964709:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 112 00000100:00100000:2.0:1787784638.964719:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668328832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784638.964762:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784638.964939:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668328832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 221us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784638.964948:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 112 00000100:00080000:1.0:1787784638.964992:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784638.965007:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668328832:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784639.256577:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.256638:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.256670:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.256691:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.256710:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.256731:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.256758:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.256790:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657961728 00000100:00100000:0.0:1787784639.256801:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657961728 00000100:00100000:0.0:1787784639.256871:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657962240 00000100:00100000:0.0:1787784639.256874:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657962240 00000100:00100000:0.0:1787784639.256889:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657961856 00000100:00100000:0.0:1787784639.256891:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657961856 00000100:00100000:0.0:1787784639.256903:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657962112 00000100:00100000:0.0:1787784639.256905:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657962112 00000100:00100000:0.0:1787784639.256924:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:0.0:1787784639.256935:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657961728:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784639.257031:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657961728:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 98us (392us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.257039:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:0.0:1787784639.257066:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:0.0:1787784639.257072:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657962240:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784639.257121:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657962240:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.257128:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:0.0:1787784639.257142:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:0.0:1787784639.257183:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657961856:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784639.257207:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657961856:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.257214:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:0.0:1787784639.257227:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:0.0:1787784639.257234:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657962112:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784639.257257:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2354:x1874627657962112:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.257262:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:0.0:1787784639.257282:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657961984 00000100:00100000:0.0:1787784639.257286:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657961984 00000100:00100000:0.0:1787784639.257317:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657961600 00000100:00100000:0.0:1787784639.257320:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657961600 00000100:00100000:0.0:1787784639.257342:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 254 00000100:00100000:0.0:1787784639.257349:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+27:2354:x1874627657961984:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784639.257380:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+27:2354:x1874627657961984:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (710us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.257386:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 254 00000100:00100000:0.0:1787784639.257399:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 255 00000100:00100000:0.0:1787784639.257405:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:2354:x1874627657961600:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784639.257440:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:2354:x1874627657961600:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 35us (728us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.257446:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 255 00000100:00080000:0.0:1787784639.257482:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784639.257488:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784639.257491:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784639.257853:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657961472 00000100:00100000:3.0:1787784639.257860:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657961472 00000100:00100000:3.0:1787784639.257901:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 339 00000100:00100000:3.0:1787784639.257909:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657961472:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784639.257957:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657961472:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (1396us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784639.257963:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 339 00000001:02000400:1.0:1787784639.270353:0:16406:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10c: Check forbidden archive ========== 18:50:37 (1787784637) 00010000:00010000:0.0:1787784639.409529:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784639.581286:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784639.581379:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657962496 00000100:00100000:1.0:1787784639.581392:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657962496 00000100:00100000:1.0:1787784639.581475:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 613 00000100:00100000:1.0:1787784639.581498:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19913:x1874627657962496:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784639.581525:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784639.581555:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4375 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06d6 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.581665:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.581683:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.581696:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.581723:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.581730:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.581777:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583364:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc438a lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583388:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc438a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583400:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc438a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784639.583414:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:1.0:1787784639.583576:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583585:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583591:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583598:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.583603:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583623:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583638:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:1.0:1787784639.583642:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.583655:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583663:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc437c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583678:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583684:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583689:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583695:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010aca00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.583698:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583704:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583712:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff9895029d1900) 00010000:00010000:1.0:1787784639.583714:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.583717:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.583724:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4383 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.584261:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4375 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06d6 expref: 30 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.584290:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc438a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06d6 expref: 29 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.584299:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784639.584301:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.584304:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac200, rc 0) 00000100:00100000:1.0:1787784639.584403:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19913:x1874627657962496:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2916us (3131us total) trans 4294967713 rc 0/0 00000100:00100000:1.0:1787784639.584415:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 613 00000100:00100000:1.0:1787784639.591044:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.591203:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657962624 00000100:00100000:0.0:1787784639.591216:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657962624 00000100:00100000:0.0:1787784639.591285:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 614 00000100:00100000:0.0:1787784639.591297:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19913:x1874627657962624:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1787784639.591440:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19913:x1874627657962624:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 145us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.591448:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 614 00000100:00100000:0.0:1787784639.603423:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.603543:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657962752 00000100:00100000:2.0:1787784639.603556:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657962752 00000100:00100000:2.0:1787784639.603645:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 23 00000100:00100000:2.0:1787784639.603656:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2357:x1874627657962752:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784639.603677:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784639.603711:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc438a lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06d6 expref: 29 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.603737:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784639.603743:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784639.603821:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+28:2357:x1874627657962752:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 166us (411us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784639.603829:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 23 00000100:00100000:3.0:1787784639.611996:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.612127:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963008 00000100:00100000:0.0:1787784639.612139:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963008 00000100:00100000:0.0:1787784639.612248:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 615 00000100:00100000:0.0:1787784639.612259:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:19914:x1874627657963008:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:0.0:1787784639.612340:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784639.612378:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4210 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b064a expref: 62 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.612401:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:0.0:1787784639.612408:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784639.612773:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.612790:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 27 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.612799:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.612840:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 27 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.612858:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 27 pid: 5811 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784639.612874:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668328960:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784639.613002:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7aee40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784639.617588:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.617677:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963136 00000100:00100000:2.0:1787784639.617690:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963136 00010000:00010000:2.0:1787784639.617742:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 28 pid: 5811 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784639.617788:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 180 00000100:00100000:2.0:1787784639.617800:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+28:2356:x1874627657963136:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784639.617838:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784639.617848:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 28 pid: 5811 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.617864:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 28 pid: 5811 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.617894:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:2.0:1787784639.617901:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.617912:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.617920:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.617943:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784639.618011:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+28:2356:x1874627657963136:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 213us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784639.618020:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 180 00010000:00010000:0.0:1787784639.618396:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655a00/0xb0ce63137acc424f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b065f expref: 27 pid: 5811 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784639.618416:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668328960:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784639.618442:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.618454:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.618460:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.618952:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.618977:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.618990:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784639.620123:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x26:0x0]: returning size 512 00010000:00010000:0.0:1787784639.620138:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.620177:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.620184:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.620189:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbcc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.620193:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.620200:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.620212:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542000) 00010000:00010000:0.0:1787784639.620215:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.620218:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.620234:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4398 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784639.620254:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004f966cc3 reply state 000000001ffb7a51 transno 4294967714 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784639.620263:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784639.620358:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19914:x1874627657963008:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 8096us (8372us total) trans 4294967714 rc 0/0 00000100:00100000:0.0:1787784639.620367:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 615 00010000:00010000:3.0:1787784639.626627:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.626650:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.626660:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.626667:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784639.626674:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.626683:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.626704:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x843a253e (ffff98950a162d00) 00010000:00010000:3.0:1787784639.626707:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784639.626713:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.626744:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc439f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784639.630560:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.630999:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963264 00000100:00100000:0.0:1787784639.631015:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963264 00000100:00100000:1.0:1787784639.631270:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 616 00000100:00100000:1.0:1787784639.631286:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+62:19915:x1874627657963264:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784639.631323:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784639.631355:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc43a6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06dd expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.631446:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc43ad lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.631464:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc43ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.631477:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc43ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784639.631557:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784639.631577:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 16 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784639.631623:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc43ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.631653:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc43a6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06dd expref: 64 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.631688:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc43ad lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06dd expref: 63 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.631699:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:1.0:1787784639.631703:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784639.631708:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac800, rc 0) 00000100:00100000:1.0:1787784639.631790:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19915:x1874627657963264:12345-192.168.203.43@tcp:101:bash.0 Request processed in 506us (1244us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784639.631798:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 616 00000100:00100000:3.0:1787784639.635639:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.635785:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963392 00000100:00100000:2.0:1787784639.635798:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963392 00000100:00100000:2.0:1787784639.635875:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 617 00000100:00100000:2.0:1787784639.635887:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+63:19915:x1874627657963392:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784639.635913:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784639.635941:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc43b4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06e4 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636144:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636185:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636199:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636229:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636239:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636249:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636399:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc43c9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636407:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc43c9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636413:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc43c9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784639.636451:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x26:0x0]: returning size 512 00000004:00200000:2.0:1787784639.636455:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784639.636480:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc43c9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636504:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636511:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636518:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636525:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784639.636530:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636539:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636553:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:2.0:1787784639.636556:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784639.636562:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636567:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc43bb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636579:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636586:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636591:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636618:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784639.636622:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636629:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636639:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x843a253e (ffff989531111b00) 00010000:00010000:2.0:1787784639.636641:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784639.636644:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636651:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc43c2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636689:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc43b4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06e4 expref: 65 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636703:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099600/0xb0ce63137acc43c9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06e4 expref: 64 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.636711:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:2.0:1787784639.636713:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784639.636715:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401099600, rc 0) 00000100:00100000:2.0:1787784639.636787:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+64:19915:x1874627657963392:12345-192.168.203.43@tcp:101:bash.0 Request processed in 901us (1161us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784639.636795:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 617 00000100:00100000:3.0:1787784639.639830:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.640309:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963520 00000100:00100000:0.0:1787784639.640322:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963520 00000100:00100000:0.0:1787784639.640399:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 618 00000100:00100000:0.0:1787784639.640410:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+64:19915:x1874627657963520:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784639.640437:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784639.640463:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc43d0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06eb expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640541:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640558:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640573:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640622:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640631:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640638:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640730:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640739:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640746:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640753:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.640758:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640768:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640782:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:0.0:1787784639.640785:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.640790:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640796:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc43d7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640817:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640823:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640829:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640834:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.640837:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640843:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640852:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff98950a162000) 00010000:00010000:0.0:1787784639.640854:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.640858:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640863:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43de lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640903:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc43d0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06eb expref: 65 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640915:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:0.0:1787784639.640917:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.640926:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc43d0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06eb expref: 65 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.640936:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbcc00, rc 0) 00000100:00100000:0.0:1787784639.641026:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+64:19915:x1874627657963520:12345-192.168.203.43@tcp:101:bash.0 Request processed in 617us (1211us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784639.641034:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 618 00000100:00100000:3.0:1787784639.650651:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784639.650839:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963648 00000100:00100000:1.0:1787784639.650851:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963648 00000100:00100000:1.0:1787784639.650928:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 619 00000100:00100000:1.0:1787784639.650941:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19916:x1874627657963648:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784639.650968:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784639.650994:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc43e5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06f2 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.651077:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.651094:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.651130:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784639.651360:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784639.651365:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 16 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784639.651391:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.651417:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc43e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 29 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.651437:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 28 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.651445:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:1.0:1787784639.651448:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784639.651451:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad800, rc 0) 00000100:00100000:1.0:1787784639.651523:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+28:19916:x1874627657963648:12345-192.168.203.43@tcp:101:rm.0 Request processed in 585us (886us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784639.651530:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 619 00000100:00100000:3.0:1787784639.654757:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.654942:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963776 00000100:00100000:0.0:1787784639.654956:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963776 00000100:00100000:0.0:1787784639.655029:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 620 00000100:00100000:0.0:1787784639.655041:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+28:19916:x1874627657963776:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784639.655066:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784639.655092:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc43f3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b06f9 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655214:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655232:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655247:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655264:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655272:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655279:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655455:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc4408 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655464:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc4408 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655473:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc4408 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784639.655513:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x26:0x0]: returning size 512 00000004:00200000:0.0:1787784639.655517:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784639.655545:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc4408 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655559:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655565:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655572:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655579:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.655583:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655592:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655621:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:0.0:1787784639.655631:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.655637:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655644:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc43fa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655663:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655669:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655676:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655681:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.655684:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655691:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655699:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x843a253e (ffff98950a162000) 00010000:00010000:0.0:1787784639.655701:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.655704:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655710:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4401 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655749:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc43f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f9 expref: 30 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655776:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd200/0xb0ce63137acc4408 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f9 expref: 29 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.655784:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:0.0:1787784639.655787:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784639.655789:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd200, rc 0) 00000100:00100000:0.0:1787784639.655875:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19916:x1874627657963776:12345-192.168.203.43@tcp:101:rm.0 Request processed in 836us (1130us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.655884:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 620 00000100:00100000:3.0:1787784639.661515:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.661653:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657963904 00000100:00100000:2.0:1787784639.661686:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657963904 00000100:00100000:2.0:1787784639.661763:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 621 00000100:00100000:2.0:1787784639.661775:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19916:x1874627657963904:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784639.661802:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784639.661830:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc440f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0700 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.661923:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.661942:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.661956:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.661974:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.661982:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.661988:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662072:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662082:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662089:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662101:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784639.662107:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662115:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662130:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:2.0:1787784639.662133:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784639.662138:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662193:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4416 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662211:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662216:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662222:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662227:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784639.662230:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662237:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662258:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff989531111b00) 00010000:00010000:2.0:1787784639.662260:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784639.662263:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662270:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc441d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662311:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc440f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0700 expref: 30 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662322:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:2.0:1787784639.662325:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784639.662328:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc440f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0700 expref: 30 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.662338:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098200, rc 0) 00000100:00100000:2.0:1787784639.662406:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19916:x1874627657963904:12345-192.168.203.43@tcp:101:rm.0 Request processed in 634us (906us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784639.662414:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 621 00000100:00100000:0.0:1787784639.671443:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784639.671560:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657964032 00000100:00100000:1.0:1787784639.671573:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657964032 00000100:00100000:1.0:1787784639.671661:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 622 00000100:00100000:1.0:1787784639.671673:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19916:x1874627657964032:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784639.671701:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784639.671727:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4424 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0707 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.671857:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.671876:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.671889:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.671906:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.671912:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.671919:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672002:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672009:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672016:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672022:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.672027:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672045:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672060:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:1.0:1787784639.672064:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.672069:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672076:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc442b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672091:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672097:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672103:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672109:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.672112:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672119:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672125:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff9895029d1c00) 00010000:00010000:1.0:1787784639.672128:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.672131:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672218:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc4432 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672261:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4424 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0707 expref: 30 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672273:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:1.0:1787784639.672276:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.672280:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4424 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0707 expref: 30 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.672291:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010aca00, rc 0) 00000100:00100000:1.0:1787784639.672360:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19916:x1874627657964032:12345-192.168.203.43@tcp:101:rm.0 Request processed in 689us (933us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784639.672368:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 622 00000100:00100000:3.0:1787784639.680978:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.681082:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657964160 00000100:00100000:2.0:1787784639.681095:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657964160 00000100:00100000:1.0:1787784639.681329:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 623 00000100:00100000:1.0:1787784639.681347:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19917:x1874627657964160:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784639.681378:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784639.681409:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4439 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b070e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681511:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681531:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681568:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681591:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681600:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681619:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681715:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681724:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681731:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681737:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.681742:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681751:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681766:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:1.0:1787784639.681770:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.681777:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681791:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc4440 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681809:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681815:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681821:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681826:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.681830:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681836:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681845:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff9895029d1e00) 00010000:00010000:1.0:1787784639.681847:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.681851:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681856:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc4447 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681909:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4439 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b070e expref: 30 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681935:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:1.0:1787784639.681937:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.681942:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4439 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b070e expref: 30 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.681953:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac600, rc 0) 00000100:00100000:1.0:1787784639.682096:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19917:x1874627657964160:12345-192.168.203.43@tcp:101:cp.0 Request processed in 755us (1132us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784639.682104:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 623 00000100:00100000:0.0:1787784639.691435:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784639.691722:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657964288 00000100:00100000:1.0:1787784639.691737:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657964288 00000100:00100000:1.0:1787784639.691821:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 624 00000100:00100000:1.0:1787784639.691834:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19917:x1874627657964288:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784639.691863:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784639.691889:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc444e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0715 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.691976:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.691994:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692007:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692024:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692050:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692057:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692142:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692172:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692179:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692186:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.692190:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692198:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692212:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:1.0:1787784639.692215:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.692221:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692228:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4455 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692242:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692255:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692261:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692266:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.692270:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692276:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692285:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff9895029d1c00) 00010000:00010000:1.0:1787784639.692287:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.692290:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692296:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc445c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692336:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc444e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0715 expref: 30 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692348:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:1.0:1787784639.692350:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.692353:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc444e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0715 expref: 30 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.692373:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539134a00, rc 0) 00000100:00100000:1.0:1787784639.692457:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19917:x1874627657964288:12345-192.168.203.43@tcp:101:cp.0 Request processed in 626us (1036us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784639.692465:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 624 00000100:00100000:0.0:1787784639.703926:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.704219:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657964544 00000100:00100000:2.0:1787784639.704233:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657964544 00000100:00100000:2.0:1787784639.704310:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 625 00000100:00100000:2.0:1787784639.704322:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:19917:x1874627657964544:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784639.704351:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784639.704400:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc4463 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b071c expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.704583:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.705017:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.705034:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.705053:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.705061:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.705067:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784639.710275:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xd:0x0], index 0: 0 00010000:00010000:2.0:1787784639.710947:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.710974:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.710994:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711049:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711082:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711090:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711097:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784639.711101:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711110:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711129:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:2.0:1787784639.711132:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784639.711320:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711330:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc446a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784639.711348:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e8bcbd2c reply state 000000007ab0199d transno 4294967715 ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784639.711359:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784639.711761:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098200/0xb0ce63137acc4463 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 32 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711780:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 31 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.711804:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784639.711808:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784639.711811:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401099c00, rc 0) 00000100:00100000:2.0:1787784639.713259:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19917:x1874627657964544:12345-192.168.203.43@tcp:101:cp.0 Request processed in 8926us (9334us total) trans 4294967715 rc 0/0 00000100:00100000:2.0:1787784639.713274:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 625 00000100:00100000:3.0:1787784639.721439:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.722220:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657964672 00000100:00100000:0.0:1787784639.722235:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657964672 00000100:00100000:0.0:1787784639.722306:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 161 00000100:00100000:0.0:1787784639.722318:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+7:2356:x1874627657964672:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784639.722342:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784639.722367:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0723 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784639.722401:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784639.722427:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=430s v=(0 0) 00010000:00010000:0.0:1787784639.722433:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784639.722446:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 8 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.722494:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 8 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.722511:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd400, rc 0) 00000100:00100000:0.0:1787784639.722619:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657964672:12345-192.168.203.43@tcp:101:cp.0 Request processed in 292us (1185us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.722627:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 161 00010000:00010000:1.0:1787784639.729042:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.729065:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.729075:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.729082:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401098e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784639.729087:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.729096:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.729118:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff989531111b00) 00010000:00010000:1.0:1787784639.729122:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784639.729127:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784639.729134:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc4471 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784639.731585:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.731747:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657964800 00000100:00100000:2.0:1787784639.731760:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657964800 00000100:00100000:2.0:1787784639.731866:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 162 00000100:00100000:2.0:1787784639.731879:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:19917:x1874627657964800:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784639.731904:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784639.731929:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b20da214/0xb0ce63137acc448d lrc: 2/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b072a expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.731967:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.731978:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784639.731982:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784639.731988:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b20da214/0xb0ce63137acc448d lrc: 4/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b072a expref: 9 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.732028:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.732071:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 9 pid: 10688 timeout: 530 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784639.732088:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668329088:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784639.732228:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508c67840 going to sleep for 11 seconds 00000100:00100000:0.0:1787784639.747014:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.747136:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668329088:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784639.747224:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b20da214/0xb0ce63137acc448d lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b072a expref: 9 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.747261:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098200, rc 0) 00000100:00100000:2.0:1787784639.747332:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:19917:x1874627657964800:12345-192.168.203.43@tcp:101:cp.0 Request processed in 15456us (15826us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784639.747344:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 162 00000100:00100000:3.0:1787784639.748023:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657964928 00000100:00100000:3.0:1787784639.748034:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657964928 00010000:00010000:3.0:1787784639.748074:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 9 pid: 10688 timeout: 530 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784639.748103:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:3.0:1787784639.748112:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657964928:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784639.748129:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784639.748186:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 9 pid: 10688 timeout: 530 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.748211:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 9 pid: 10688 timeout: 530 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.748240:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024aa1542/0xb0ce63137acc4486 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0723 expref: 9 pid: 10688 timeout: 530 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.748283:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784639.748288:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784639.748299:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b20da214/0xb0ce63137acc448d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b072a expref: 8 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.748312:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b20da214/0xb0ce63137acc448d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b072a expref: 8 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.748354:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000b20da214/0xb0ce63137acc448d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b072a expref: 8 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784639.748371:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668329216:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784639.748448:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cfe600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784639.753626:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668329216:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784639.753673:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784639.753756:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657964928:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5643us (6756us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784639.753773:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:1.0:1787784639.761954:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784639.763046:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657965184 00000100:00100000:3.0:1787784639.763064:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657965184 00000100:00100000:3.0:1787784639.763172:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 130 00000100:00100000:3.0:1787784639.763184:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:19917:x1874627657965184:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:3.0:1787784639.763890:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+32:19917:x1874627657965184:12345-192.168.203.43@tcp:35:cp.0 Request processed in 704us (1944us total) trans 4294967716 rc 0/0 00000100:00100000:3.0:1787784639.763919:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 130 00000100:00100000:1.0:1787784639.810865:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.810999:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657965440 00000100:00100000:0.0:1787784639.811013:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657965440 00000100:00100000:0.0:1787784639.811093:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 626 00000100:00100000:0.0:1787784639.811104:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+64:19920:x1874627657965440:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784639.811132:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784639.811205:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4494 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0731 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811310:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811328:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811342:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811360:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811387:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811395:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811574:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811585:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811593:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784639.811619:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x10:0x0]: returning size 158 00010000:00010000:0.0:1787784639.811748:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811757:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811764:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811771:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbca00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.811776:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811785:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811811:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:0.0:1787784639.811814:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.811820:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811826:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc449b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811843:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811850:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811856:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811861:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbda00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.811864:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811870:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811877:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff98950a162d00) 00010000:00010000:0.0:1787784639.811879:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.811882:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811888:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811948:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4494 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 67 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811963:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 66 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.811971:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff989519d44d00) 00010000:00010000:0.0:1787784639.811973:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784639.811976:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbde00, rc 0) 00000100:00100000:0.0:1787784639.812048:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:19920:x1874627657965440:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 946us (1198us total) trans 4294967717 rc 0/0 00000100:00100000:0.0:1787784639.812057:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 626 00000100:00100000:1.0:1787784639.815352:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784639.815460:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657965696 00000100:00100000:3.0:1787784639.815473:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657965696 00000100:00100000:3.0:1787784639.815543:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 627 00000100:00100000:3.0:1787784639.815554:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:19920:x1874627657965696:12345-192.168.203.43@tcp:55:lfs.0 00010000:00010000:3.0:1787784639.815658:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.815678:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 31 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.815687:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 66 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.815695:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 5/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.815754:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 66 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.815769:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 66 pid: 5812 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784639.815787:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668329344:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784639.815901:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 31 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.815933:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 31 pid: 5813 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784639.815943:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539252d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668329472:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784639.815972:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cfe600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784639.833383:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668329344:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784639.833539:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539252d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668329472:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784639.833577:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784639.833727:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784639.833760:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.833844:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657965952 00000100:00100000:2.0:1787784639.833855:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657965952 00010000:00010000:2.0:1787784639.833907:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 32 pid: 5813 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784639.833945:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657965824 00000100:00100000:2.0:1787784639.833949:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657965824 00010000:00010000:2.0:1787784639.833963:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 67 pid: 5812 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784639.833997:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:2.0:1787784639.834007:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+32:2357:x1874627657965952:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784639.834043:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784639.834051:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 32 pid: 5813 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834061:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 32 pid: 5813 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834085:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc4478 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b071c expref: 32 pid: 5813 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834099:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784639.834103:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834111:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834115:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784639.834222:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2357:x1874627657965952:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 215us (496us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784639.834230:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:2.0:1787784639.834249:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 183 00000100:00100000:2.0:1787784639.834256:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:2357:x1874627657965824:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784639.834263:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784639.834268:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 67 pid: 5812 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834290:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 67 pid: 5812 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834303:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc44a9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0731 expref: 67 pid: 5812 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834314:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784639.834318:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834325:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834334:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.834356:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784639.834377:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:2357:x1874627657965824:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 121us (616us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784639.834384:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 183 00010000:00010000:3.0:1787784639.834805:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.834818:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.837015:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.837033:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.837041:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.837048:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784639.837053:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.837078:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.837099:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff989519d44d00) 00010000:00010000:3.0:1787784639.837102:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787784639.837108:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784639.837114:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc44b7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784639.837267:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:19920:x1874627657965696:12345-192.168.203.43@tcp:55:lfs.0 Request processed in 21708us (21934us total) trans 4294967718 rc 0/0 00000100:00100000:3.0:1787784639.837279:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 627 00000100:00100000:1.0:1787784639.840037:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784639.840132:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657966208 00000100:00100000:3.0:1787784639.840177:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657966208 00000100:00100000:3.0:1787784639.840247:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 131 00000100:00100000:3.0:1787784639.840258:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507679450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:19920:x1874627657966208:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:3.0:1787784639.840977:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507679450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:19920:x1874627657966208:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 715us (948us total) trans 4294967719 rc 0/0 00000100:00100000:3.0:1787784639.840990:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 131 00000100:00100000:1.0:1787784639.863136:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.863325:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657966336 00000100:00100000:0.0:1787784639.863340:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657966336 00000100:00100000:0.0:1787784639.863420:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 628 00000100:00100000:0.0:1787784639.863432:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:19921:x1874627657966336:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784639.863469:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784639.863503:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc44be lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0738 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863592:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863622:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863636:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863679:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863688:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863695:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863831:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc44d3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863838:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc44d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863843:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc44d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784639.863878:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x10:0x0]: returning size 158 00000004:00200000:0.0:1787784639.863882:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784639.863902:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc44d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863912:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc44d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863919:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863924:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863937:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863942:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbda00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.863945:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863952:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863965:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:0.0:1787784639.863968:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.863972:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863977:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc44c5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863989:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863993:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.863996:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.864000:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbca00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784639.864002:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.864012:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.864017:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff98950a162d00) 00010000:00010000:0.0:1787784639.864019:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784639.864020:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.864023:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc44cc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.864056:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc44be lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0738 expref: 69 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.864066:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc800/0xb0ce63137acc44d3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0738 expref: 68 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.864072:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff98950a162f00) 00010000:00010000:0.0:1787784639.864074:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784639.864076:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc800, rc 0) 00000100:00100000:0.0:1787784639.864224:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:19921:x1874627657966336:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 793us (1101us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784639.864233:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 628 00000100:00100000:1.0:1787784639.868568:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784639.868872:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657966464 00000100:00100000:0.0:1787784639.868886:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657966464 00000100:00100000:0.0:1787784639.869019:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 163 00000100:00100000:0.0:1787784639.869031:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527def050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657966464:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784639.869079:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784639.869103:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007c524efc/0xb0ce63137acc44da lrc: 2/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b073f expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784639.869239:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668329600:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784639.869364:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5180 going to sleep for 11 seconds 00002000:00010000:0.0:1787784639.872733:0:13825:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1787784639.872750:0:13825:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb mtime: 0 -> 1787784639 00002000:00010000:0.0:1787784639.872753:0:13825:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb atime: 0 -> 1787784639 00002000:00010000:0.0:1787784639.872755:0:13825:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb ctime: 0 -> 1787784639 00000100:00100000:0.0:1787784639.872788:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668329600:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784639.872846:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007c524efc/0xb0ce63137acc44da lrc: 1/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b073f expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.872879:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007c524efc/0xb0ce63137acc44da lrc: 0/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b073f expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784639.872896:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd000, rc 0) 00000100:00100000:0.0:1787784639.872986:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527def050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657966464:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3956us (4432us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784639.872996:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 163 00000100:00100000:1.0:1787784639.878066:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.878352:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657966592 00000100:00100000:2.0:1787784639.878367:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657966592 00000100:00100000:2.0:1787784639.878447:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 629 00000100:00100000:2.0:1787784639.878461:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:19921:x1874627657966592:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784639.878489:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784639.878519:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44e1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0746 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.878790:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc44e8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.878817:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc44e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.878836:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc44e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784639.878930:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x26:0x0]: returning size 512 00000004:00200000:2.0:1787784639.878935:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784639.878964:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc44e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.879003:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44e1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0746 expref: 70 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.879026:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401098000/0xb0ce63137acc44e8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0746 expref: 69 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.879037:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff989531111f00) 00010000:00010000:2.0:1787784639.879040:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784639.879045:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098000, rc 0) 00000100:00100000:2.0:1787784639.882269:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:19921:x1874627657966592:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 690us (1102us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784639.882289:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 629 00000100:00100000:3.0:1787784639.882748:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.882900:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657966720 00000100:00100000:2.0:1787784639.882912:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657966720 00000100:00100000:2.0:1787784639.884336:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 164 00000100:00100000:2.0:1787784639.884354:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627657966720:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784639.884385:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784639.884409:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e43d2ecf/0xb0ce63137acc44ef lrc: 2/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b074d expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784639.884492:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668329728:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784639.884620:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508c67840 going to sleep for 11 seconds 00000100:00100000:2.0:1787784639.887346:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668329728:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784639.887414:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e43d2ecf/0xb0ce63137acc44ef lrc: 1/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b074d expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.887454:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e43d2ecf/0xb0ce63137acc44ef lrc: 0/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b074d expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784639.887474:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401098e00, rc 0) 00000100:00100000:2.0:1787784639.887558:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627657966720:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3206us (4811us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784639.887571:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 164 00000100:00100000:1.0:1787784639.892121:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784639.892776:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657966976 00000100:00100000:2.0:1787784639.892791:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657966976 00000100:00100000:2.0:1787784639.892873:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 630 00000100:00100000:2.0:1787784639.892885:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:19921:x1874627657966976:12345-192.168.203.43@tcp:58:lfs.0 00000100:00100000:2.0:1787784639.893076:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:19921:x1874627657966976:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 192us (968us total) trans 0 rc -1/-1 00000100:00100000:2.0:1787784639.893107:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 630 00000040:00080000:0.0:1787784640.369532:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784640.369546:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784640.370108:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784640.370123:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784640.433476:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784641.012565:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668329984:0@lo:400:kworker.0 00000100:00100000:3.0:1787784641.012652:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784641.012718:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784641.012726:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530505680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668329856:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784641.012770:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784641.012815:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668329984 00000100:00100000:3.0:1787784641.012822:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668329984 00000100:00100000:3.0:1787784641.012866:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:3.0:1787784641.012875:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668329984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784641.012929:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668329984:12345-0@lo:400:kworker.0 Request processed in 55us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784641.012936:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:3.0:1787784641.012971:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668329856 00000100:00100000:3.0:1787784641.012975:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668329856 00000100:00100000:3.0:1787784641.013015:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:3.0:1787784641.013022:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668329856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784641.013051:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784641.013087:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668329856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (316us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784641.013093:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00080000:3.0:1787784641.013229:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784641.013356:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530505680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668329856:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784641.013367:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668329984:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.015768:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668330112:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.015860:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784641.015902:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784641.015909:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b11680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668330240:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.015935:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784641.015971:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668330112 00000100:00100000:2.0:1787784641.015978:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668330112 00000100:00100000:2.0:1787784641.016022:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668330240 00000100:00100000:2.0:1787784641.016026:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668330240 00000100:00100000:2.0:1787784641.016060:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:2.0:1787784641.016069:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668330112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016126:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668330112:12345-0@lo:400:kworker.0 Request processed in 58us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784641.016134:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:2.0:1787784641.016181:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1787784641.016190:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668330240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016227:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668330240:12345-0@lo:400:kworker.0 Request processed in 39us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784641.016234:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00080000:2.0:1787784641.016259:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784641.016287:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b11680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668330240:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016298:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668330112:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016339:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668330368:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016382:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784641.016405:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784641.016412:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668330496:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016438:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784641.016465:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668330368 00000100:00100000:2.0:1787784641.016468:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668330368 00000100:00100000:2.0:1787784641.016501:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1787784641.016509:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668330368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016545:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668330368:12345-0@lo:400:kworker.0 Request processed in 38us (163us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784641.016551:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00080000:2.0:1787784641.016571:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784641.016587:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668330368:0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016618:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668330496 00000100:00100000:2.0:1787784641.016621:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668330496 00000100:00100000:2.0:1787784641.016648:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:2.0:1787784641.016655:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668330496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784641.016692:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668330496:12345-0@lo:400:kworker.0 Request processed in 37us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784641.016698:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00080000:2.0:1787784641.016714:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784641.016731:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668330496:0@lo:400:kworker.0 00000040:00080000:1.0:1787784641.393963:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784641.393976:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784641.394875:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 2 00000040:00080000:1.0:1787784641.394887:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784641.457541:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784642.485767:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784642.619812:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967717 is committed 00000001:00080000:3.0:1787784642.623675:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967713 is committed 00000001:00080000:3.0:1787784642.623692:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967715 is committed 00000001:00080000:3.0:1787784642.623698:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967719 is committed 00000001:00080000:3.0:1787784642.623705:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967716 is committed 00010000:00010000:0.0:1787784643.505400:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784644.081495:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668330624:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784644.081611:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784644.081685:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668330624 00000100:00100000:0.0:1787784644.081701:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668330624 00000100:00100000:0.0:1787784644.081794:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 114 00000100:00100000:0.0:1787784644.081807:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668330624:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784644.081852:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784644.093502:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668330624:12345-0@lo:13:osp-pre-0-0.0 Request processed in 11687us (11881us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784644.093517:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 114 00000100:00080000:0.0:1787784644.093844:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784644.096501:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668330624:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784644.208233:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784644.216370:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657967104 00000100:00100000:2.0:1787784644.216392:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657967104 00000100:00080000:1.0:1787784644.216477:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784644.216507:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 631 00000100:00100000:2.0:1787784644.216524:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19913:x1874627657967104:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1787784644.216958:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19913:x1874627657967104:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 438us (8738us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784644.216968:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 631 00000100:00100000:1.0:1787784644.225543:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784644.225949:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657967360 00000100:00100000:2.0:1787784644.225963:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657967360 00000100:00100000:2.0:1787784644.226035:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 132 00000100:00100000:2.0:1787784644.226046:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+27:19913:x1874627657967360:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784644.226730:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+28:19913:x1874627657967360:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 680us (1198us total) trans 4294967720 rc 0/0 00000100:00100000:2.0:1787784644.228032:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 132 00000100:00100000:1.0:1787784644.371969:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784644.372028:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784644.372050:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784644.372089:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784644.372133:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784644.372199:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784644.372238:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657967488 00000100:00100000:1.0:1787784644.372251:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657967488 00000100:00100000:1.0:1787784644.372317:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:1.0:1787784644.372329:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657967488:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784644.372421:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657967488:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 94us (329us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784644.372430:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:1.0:1787784644.372495:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657967872 00000100:00100000:1.0:1787784644.372500:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657967872 00000100:00100000:1.0:1787784644.372542:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 198 00000100:00100000:1.0:1787784644.372549:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657967872:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784644.372595:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657967872:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 47us (567us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784644.372601:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 198 00000100:00100000:1.0:1787784644.372611:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657968128 00000100:00100000:1.0:1787784644.372614:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657968128 00000100:00100000:1.0:1787784644.372628:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657967744 00000100:00100000:1.0:1787784644.372631:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657967744 00000100:00100000:1.0:1787784644.372643:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657968000 00000100:00100000:1.0:1787784644.372645:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657968000 00000100:00100000:1.0:1787784644.372656:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 199 00000100:00100000:1.0:1787784644.372661:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657968128:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784644.372685:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657968128:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (635us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784644.372691:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 199 00000100:00100000:1.0:1787784644.372701:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 200 00000100:00100000:1.0:1787784644.372707:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657967744:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784644.372730:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657967744:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784644.372736:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 200 00000100:00100000:1.0:1787784644.372748:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 201 00000100:00100000:1.0:1787784644.372754:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657968000:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784644.372776:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657968000:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (577us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784644.372789:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 201 00000100:00080000:1.0:1787784644.372815:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784644.372820:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784644.375198:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657967616 00000100:00100000:2.0:1787784644.375221:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657967616 00000100:00100000:2.0:1787784644.375330:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 259 00000100:00100000:2.0:1787784644.375343:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+65:2354:x1874627657967616:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784644.375446:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+65:2354:x1874627657967616:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 106us (3492us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784644.375455:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 259 00000100:00080000:1.0:1787784644.376534:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784644.529481:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784645.567593:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787784645.978089:0:16602:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 18:50:43 (1787784643) 00000100:00100000:2.0:1787784646.130917:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668330752:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.131029:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784646.131090:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668331264:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.131123:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784646.132228:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784646.132252:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668331136:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.132307:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784646.132371:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668330880:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.132419:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784646.132442:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668331392:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.132464:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784646.132484:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784646.132490:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b11680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668331008:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.132512:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784646.132530:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668331520:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784646.132555:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784646.132725:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668330752 00000100:00100000:2.0:1787784646.132734:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668330752 00000100:00100000:2.0:1787784646.132791:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1787784646.132802:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668330752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.132864:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668330752:12345-0@lo:400:kworker.0 Request processed in 63us (1835us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.132872:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1787784646.132912:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668331264 00000100:00100000:2.0:1787784646.132916:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668331264 00000100:00100000:2.0:1787784646.132961:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668330880 00000100:00100000:2.0:1787784646.132965:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668330880 00000100:00100000:2.0:1787784646.132985:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668331008 00000100:00100000:2.0:1787784646.132988:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668331008 00000100:00100000:2.0:1787784646.133010:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:2.0:1787784646.133017:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668331264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.133053:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668331264:12345-0@lo:400:kworker.0 Request processed in 37us (1929us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.133069:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:2.0:1787784646.133087:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1787784646.133094:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668330880:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.133131:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668330880:12345-0@lo:400:kworker.0 Request processed in 38us (712us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.133137:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1787784646.133178:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1787784646.133186:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668331008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.133215:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668331008:12345-0@lo:400:kworker.0 Request processed in 31us (703us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.133221:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1787784646.133249:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668331136 00000100:00100000:2.0:1787784646.133254:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668331136 00000100:00100000:2.0:1787784646.133289:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668331392 00000100:00100000:2.0:1787784646.133293:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668331392 00000100:00100000:2.0:1787784646.133316:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1787784646.133324:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668331136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.133359:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668331136:12345-0@lo:400:kworker.0 Request processed in 37us (1051us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.133366:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1787784646.133376:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:2.0:1787784646.133382:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668331392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.133675:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668331392:12345-0@lo:400:kworker.0 Request processed in 292us (1208us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.133682:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:2.0:1787784646.133718:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668331520 00000100:00100000:2.0:1787784646.133722:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668331520 00000100:00100000:2.0:1787784646.133758:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1787784646.133767:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668331520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784646.133802:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784646.133845:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668331520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (1289us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.133851:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00080000:2.0:1787784646.133880:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784646.133884:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784646.133888:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784646.133933:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b11680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668331008:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.133957:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668331520:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784646.133969:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668330880:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.133979:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668331392:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.134038:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b12580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668331136:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.134052:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668330752:0@lo:400:kworker.0 00000100:00100000:2.0:1787784646.134065:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668331264:0@lo:400:kworker.0 00000100:00100000:1.0:1787784646.228280:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.228400:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657968384 00000100:00100000:2.0:1787784646.228416:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657968384 00000100:00080000:1.0:1787784646.228512:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784646.228531:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 632 00000100:00100000:2.0:1787784646.228542:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+28:20523:x1874627657968384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784646.228586:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.229436:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc44f6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0754 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.229662:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.229681:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.229700:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.229721:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.229729:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.229736:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.230897:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc450b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.230922:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc450b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.230934:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc450b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784646.230949:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:3.0:1787784646.231138:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233000:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233025:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233032:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401099c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784646.233038:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233048:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233071:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:3.0:1787784646.233075:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.233081:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233113:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099c00/0xb0ce63137acc44fd lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233137:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233142:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233164:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233169:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784646.233172:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233178:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233188:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff989519d44600) 00010000:00010000:3.0:1787784646.233190:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.233193:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233199:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4504 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233888:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401098e00/0xb0ce63137acc44f6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0754 expref: 31 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233911:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc450b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0754 expref: 30 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.233925:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784646.233928:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.233931:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3400, rc 0) 00000100:00100000:3.0:1787784646.234046:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20523:x1874627657968384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5504us (5781us total) trans 4294967721 rc 0/0 00000100:00100000:3.0:1787784646.234058:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 632 00000100:00100000:2.0:1787784646.249877:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.249938:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.250101:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657968512 00000100:00100000:3.0:1787784646.250115:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657968512 00000100:00100000:2.0:1787784646.250142:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.250207:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657968768 00000100:00100000:3.0:1787784646.250211:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657968768 00000100:00100000:2.0:1787784646.250221:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00100000:0.0:1787784646.250282:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657968640 00000100:00100000:0.0:1787784646.250287:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657968640 00000100:00100000:3.0:1787784646.250313:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 634 00000100:00100000:2.0:1787784646.250846:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:20523:x1874627657968512:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1787784646.250995:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:20523:x1874627657968512:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 765us (1134us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.251004:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00080000:2.0:1787784646.251057:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784646.251551:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 24 00000100:00100000:0.0:1787784646.251572:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:2355:x1874627657968640:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784646.251586:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787784646.254844:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+65:20524:x1874627657968768:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784646.254884:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784646.254916:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4519 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b075b expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.255007:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.255026:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.255041:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.255062:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.255093:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.255101:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256023:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256038:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256045:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256051:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784646.256056:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256065:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256081:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:3.0:1787784646.256085:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.256090:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256096:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4520 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256111:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256127:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256133:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256138:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784646.256141:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256233:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256242:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54fab68 (ffff989505542000) 00010000:00010000:3.0:1787784646.256244:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.256247:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256253:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4527 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256309:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4519 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b075b expref: 66 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256321:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:3.0:1787784646.256323:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.256327:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4519 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b075b expref: 66 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.256336:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784646.256513:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+65:20524:x1874627657968768:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6190us (6367us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784646.256526:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 634 00010000:00010000:0.0:1787784646.260716:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc450b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0754 expref: 30 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.260757:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784646.260764:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784646.260863:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2355:x1874627657968640:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 9300us (10921us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.260875:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 24 00000100:00100000:3.0:1787784646.262234:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.262329:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969024 00000100:00100000:0.0:1787784646.262342:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969024 00000100:00100000:0.0:1787784646.262411:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 635 00000100:00100000:0.0:1787784646.262422:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+65:20524:x1874627657969024:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784646.262520:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784646.262552:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc43ad lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06dd expref: 65 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.262586:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:0.0:1787784646.262592:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784646.262931:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.262947:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 28 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.262956:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.263010:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 28 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.263028:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 28 pid: 5811 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784646.263044:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668331648:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784646.263222:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784646.270877:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.271052:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668331648:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784646.271090:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.271643:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969152 00000100:00100000:2.0:1787784646.271672:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969152 00010000:00010000:2.0:1787784646.271793:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 29 pid: 5811 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.271850:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 184 00000100:00100000:2.0:1787784646.271861:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2355:x1874627657969152:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784646.271880:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784646.271887:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 29 pid: 5811 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.271901:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 29 pid: 5811 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.271931:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc43ec lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b06f2 expref: 29 pid: 5811 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.271951:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:2.0:1787784646.271958:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.271969:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.271978:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.272011:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784646.272182:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.272194:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+28:2355:x1874627657969152:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 333us (1332us total) trans 0 rc 0/0 00010000:00010000:0.0:1787784646.272199:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.272211:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 184 00010000:00010000:0.0:1787784646.272230:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.272240:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.272249:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784646.275387:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 512 00010000:00010000:0.0:1787784646.275412:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.275424:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.275432:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.275439:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784646.275445:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.275453:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.275474:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:0.0:1787784646.275477:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784646.275493:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.275500:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc452e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784646.275528:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000025d10b2 reply state 00000000b9249dc6 transno 4294967722 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784646.275538:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784646.275665:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:20524:x1874627657969024:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 13239us (13434us total) trans 4294967722 rc 0/0 00000100:00100000:0.0:1787784646.275675:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 635 00010000:00010000:1.0:1787784646.279913:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.279938:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.279947:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.279953:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784646.279959:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.279967:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.279989:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54fab68 (ffff98950a162b00) 00010000:00010000:1.0:1787784646.279993:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784646.279999:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.280006:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4535 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784646.284823:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.285227:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969280 00000100:00100000:2.0:1787784646.285246:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969280 00000100:00100000:2.0:1787784646.285330:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 636 00000100:00100000:2.0:1787784646.285343:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+65:20525:x1874627657969280:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784646.285380:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.285422:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc453c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0762 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.285518:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4543 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.285536:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4543 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.285550:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4543 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784646.285656:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784646.285661:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 17 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784646.285691:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4543 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.285731:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc453c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0762 expref: 67 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.285754:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4543 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0762 expref: 66 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.285787:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:2.0:1787784646.285791:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784646.285795:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905000, rc 0) 00000100:00100000:2.0:1787784646.285888:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:20525:x1874627657969280:12345-192.168.203.43@tcp:101:bash.0 Request processed in 547us (1081us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.285897:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 636 00000100:00100000:3.0:1787784646.292105:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.292217:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969408 00000100:00100000:2.0:1787784646.292233:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969408 00000100:00100000:2.0:1787784646.292330:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 637 00000100:00100000:2.0:1787784646.292343:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+66:20525:x1874627657969408:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784646.292377:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.292448:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebe00/0xb0ce63137acc454a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0769 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.292540:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.292558:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.292582:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.292801:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.292810:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.292832:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.292992:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb000/0xb0ce63137acc455f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293001:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb000/0xb0ce63137acc455f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293009:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb000/0xb0ce63137acc455f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784646.293048:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 512 00000004:00200000:2.0:1787784646.293052:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784646.293090:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb000/0xb0ce63137acc455f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293107:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293114:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293121:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293128:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.293133:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293142:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293213:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:2.0:1787784646.293217:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.293224:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293231:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4551 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293245:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293250:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293256:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293262:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ebc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.293265:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293271:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293279:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54fab68 (ffff989519d44700) 00010000:00010000:2.0:1787784646.293282:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.293285:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293291:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc4558 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293346:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebe00/0xb0ce63137acc454a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0769 expref: 68 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293362:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb000/0xb0ce63137acc455f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0769 expref: 67 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.293370:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.293373:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784646.293375:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb000, rc 0) 00000100:00100000:2.0:1787784646.296029:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:20525:x1874627657969408:12345-192.168.203.43@tcp:101:bash.0 Request processed in 3672us (3923us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.296045:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 637 00000100:00100000:0.0:1787784646.301185:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.301257:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969536 00000100:00100000:2.0:1787784646.301271:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969536 00000100:00100000:2.0:1787784646.301352:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 638 00000100:00100000:2.0:1787784646.301365:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:20525:x1874627657969536:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784646.301395:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.301425:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4566 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0770 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301522:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301544:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301559:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301621:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301631:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301638:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301749:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301759:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301767:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301773:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.301778:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301787:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301801:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.301805:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.301810:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301817:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc456d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301841:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301848:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301855:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301860:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.301864:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301871:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301881:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff989530cb1500) 00010000:00010000:2.0:1787784646.301884:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.301887:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301893:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4574 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301936:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4566 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0770 expref: 68 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301948:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.301951:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.301961:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4566 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0770 expref: 68 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.301972:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea200, rc 0) 00000100:00100000:2.0:1787784646.302058:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:20525:x1874627657969536:12345-192.168.203.43@tcp:101:bash.0 Request processed in 695us (915us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784646.302067:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 638 00000100:00100000:3.0:1787784646.313740:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.313958:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969664 00000100:00100000:0.0:1787784646.313979:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969664 00000100:00100000:0.0:1787784646.314052:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00100000:0.0:1787784646.314064:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+28:20526:x1874627657969664:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784646.314302:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784646.314332:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc457b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0777 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.314430:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.314448:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.314462:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784646.314576:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784646.314581:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 17 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784646.314611:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.314643:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc457b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 30 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.314667:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 29 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.314679:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:0.0:1787784646.314682:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784646.314687:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbcc00, rc 0) 00000100:00100000:2.0:1787784646.314827:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+29:20526:x1874627657969664:12345-192.168.203.43@tcp:101:rm.0 Request processed in 763us (1103us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.314837:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00100000:3.0:1787784646.320012:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.320139:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969792 00000100:00100000:2.0:1787784646.320170:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969792 00000100:00100000:2.0:1787784646.320289:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 640 00000100:00100000:2.0:1787784646.320303:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:20526:x1874627657969792:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784646.320335:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.320365:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4589 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b077e expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320455:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320475:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320491:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320509:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320542:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320550:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320704:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebe00/0xb0ce63137acc459e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320714:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebe00/0xb0ce63137acc459e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320722:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebe00/0xb0ce63137acc459e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784646.320762:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 512 00000004:00200000:2.0:1787784646.320767:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784646.320795:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebe00/0xb0ce63137acc459e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320811:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320820:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320826:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320833:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.320838:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320854:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320869:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:2.0:1787784646.320872:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.320877:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320883:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4590 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320900:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320906:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320912:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320917:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.320920:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320926:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320932:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54fab68 (ffff989530cb1500) 00010000:00010000:2.0:1787784646.320935:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.320937:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320954:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc4597 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.320993:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4589 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b077e expref: 31 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.321007:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebe00/0xb0ce63137acc459e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b077e expref: 30 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.321015:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.321018:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784646.321021:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ebe00, rc 0) 00000100:00100000:2.0:1787784646.321114:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20526:x1874627657969792:12345-192.168.203.43@tcp:101:rm.0 Request processed in 813us (1117us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.321124:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 640 00000100:00100000:0.0:1787784646.325360:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.325506:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657969920 00000100:00100000:2.0:1787784646.325519:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657969920 00000100:00100000:2.0:1787784646.325607:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 641 00000100:00100000:2.0:1787784646.325619:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20526:x1874627657969920:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784646.325647:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.325676:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc45a5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0785 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325777:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325795:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325827:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325845:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325853:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325860:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325946:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325955:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325963:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325970:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.325975:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325983:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.325998:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.326001:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.326006:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326020:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45ac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326038:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326044:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326050:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326056:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eae00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.326059:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326066:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326074:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff989530cb1b00) 00010000:00010000:2.0:1787784646.326077:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.326080:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326086:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326127:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc45a5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0785 expref: 31 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326204:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.326207:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.326212:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc45a5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0785 expref: 31 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.326224:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ebc00, rc 0) 00000100:00100000:2.0:1787784646.326326:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20526:x1874627657969920:12345-192.168.203.43@tcp:101:rm.0 Request processed in 708us (979us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784646.326335:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 641 00000100:00100000:1.0:1787784646.330227:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.330903:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657970048 00000100:00100000:3.0:1787784646.330919:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657970048 00000100:00100000:3.0:1787784646.331001:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 642 00000100:00100000:3.0:1787784646.331013:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20526:x1874627657970048:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784646.331043:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784646.331072:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3000/0xb0ce63137acc45ba lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b078c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331527:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331547:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331664:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331684:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331692:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331794:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331883:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331891:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331898:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331904:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784646.331909:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331917:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331932:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:3.0:1787784646.331935:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.331940:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331947:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc45c1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331960:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331966:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331978:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331985:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784646.331988:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.331994:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.332004:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff989505542000) 00010000:00010000:3.0:1787784646.332006:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.332008:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.332014:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc45c8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.332056:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3000/0xb0ce63137acc45ba lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b078c expref: 31 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.332068:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:3.0:1787784646.332071:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784646.332075:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3000/0xb0ce63137acc45ba lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b078c expref: 31 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.332085:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3000, rc 0) 00000100:00100000:3.0:1787784646.332337:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20526:x1874627657970048:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1325us (2124us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784646.332355:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 642 00000100:00100000:3.0:1787784646.351971:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.353847:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657970176 00000100:00100000:2.0:1787784646.353868:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657970176 00000100:00100000:2.0:1787784646.353946:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:2.0:1787784646.353959:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20527:x1874627657970176:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784646.353986:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.354013:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc45cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0793 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354100:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354117:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354131:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354169:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354195:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354203:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354289:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354298:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354305:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354313:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eae00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.354319:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354328:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354343:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.354346:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.354352:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354359:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc45d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354375:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354387:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354393:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354398:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.354401:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354407:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354415:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff989530cb1b00) 00010000:00010000:2.0:1787784646.354418:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.354420:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354425:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc45dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354467:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc45cf lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0793 expref: 31 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354478:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.354480:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.354483:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc45cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0793 expref: 31 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.354503:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ebc00, rc 0) 00000100:00100000:2.0:1787784646.354606:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20527:x1874627657970176:12345-192.168.203.43@tcp:101:cp.0 Request processed in 649us (2649us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784646.354614:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:0.0:1787784646.357731:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.357888:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657970304 00000100:00100000:2.0:1787784646.357900:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657970304 00000100:00100000:2.0:1787784646.357970:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 644 00000100:00100000:2.0:1787784646.357984:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20527:x1874627657970304:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784646.358011:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.358039:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc45e4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b079a expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358133:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358181:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358196:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358214:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358221:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358227:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358329:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358336:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358342:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358348:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.358353:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358361:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358374:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.358377:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.358383:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358389:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc45eb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358404:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358410:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358416:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358430:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.358433:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358440:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358449:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff989530cb1500) 00010000:00010000:2.0:1787784646.358451:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.358454:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358461:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc45f2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358504:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc45e4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b079a expref: 31 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358515:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.358518:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.358522:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc45e4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b079a expref: 31 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.358532:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea200, rc 0) 00000100:00100000:2.0:1787784646.358622:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20527:x1874627657970304:12345-192.168.203.43@tcp:101:cp.0 Request processed in 641us (907us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784646.358629:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 644 00000100:00100000:3.0:1787784646.361232:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.361528:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657970560 00000100:00100000:2.0:1787784646.361558:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657970560 00000100:00100000:2.0:1787784646.361642:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 645 00000100:00100000:2.0:1787784646.361654:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20527:x1874627657970560:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784646.361680:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.361707:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc45f9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07a1 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.361886:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.361904:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.361918:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.361934:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.361943:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.361950:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784646.363318:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xf:0x0], index 1: 0 00010000:00010000:2.0:1787784646.363548:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc460e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363577:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc460e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363599:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc460e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363646:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363654:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363661:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363668:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eac00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.363673:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363682:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363698:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.363700:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.363706:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363713:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4600 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784646.363728:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000018b1a1cc reply state 000000001cf944c1 transno 4294967723 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.363737:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784646.363789:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc45f9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a1 expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363805:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ebc00/0xb0ce63137acc460e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a1 expref: 32 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.363814:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:2.0:1787784646.363816:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784646.363819:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ebc00, rc 0) 00000100:00100000:2.0:1787784646.366710:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+33:20527:x1874627657970560:12345-192.168.203.43@tcp:101:cp.0 Request processed in 5039us (5471us total) trans 4294967723 rc 0/0 00000100:00100000:2.0:1787784646.366727:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 645 00010000:00010000:0.0:1787784646.376530:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.376553:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.376576:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.376584:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784646.376590:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.376626:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.376654:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff989530cb1f00) 00010000:00010000:0.0:1787784646.376658:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784646.376664:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.376670:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4607 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784646.381242:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.381421:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657970688 00000100:00100000:0.0:1787784646.381435:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657970688 00000100:00100000:0.0:1787784646.381526:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00100000:0.0:1787784646.381535:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627657970688:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784646.381571:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784646.381605:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07a8 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784646.381645:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784646.381671:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=437s v=(0 0) 00010000:00010000:0.0:1787784646.381676:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784646.381688:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 9 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.381711:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 9 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.382334:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc600, rc 0) 00000100:00080000:2.0:1787784646.382459:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784646.382898:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627657970688:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1358us (1677us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784646.382909:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00100000:1.0:1787784646.388573:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.388799:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657970816 00000100:00100000:2.0:1787784646.388814:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657970816 00000100:00100000:2.0:1787784646.388895:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 166 00000100:00100000:2.0:1787784646.388907:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:20527:x1874627657970816:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784646.388932:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.388958:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07af expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.389000:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 10 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.389010:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784646.389015:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784646.389021:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.389091:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 10 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.389106:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 10 pid: 13825 timeout: 537 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.389455:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668331776:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784646.389655:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784646.393109:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.393486:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657970944 00000100:00100000:3.0:1787784646.393498:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657970944 00010000:00010000:3.0:1787784646.393555:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 11 pid: 13825 timeout: 537 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784646.393695:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 185 00000100:00100000:3.0:1787784646.393706:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657970944:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784646.393727:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784646.393756:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 11 pid: 13825 timeout: 537 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.393772:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 11 pid: 13825 timeout: 537 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.393805:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784646.393809:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784646.393820:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.393857:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.393908:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784646.393924:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668331904:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784646.394029:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895012549c0 going to sleep for 11 seconds 00010000:00010000:2.0:1787784646.395520:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fb4e0bce/0xb0ce63137acc461c lrc: 0/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07a8 expref: 11 pid: 13825 timeout: 537 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.395546:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668331776:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784646.395607:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.395623:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eaa00, rc 0) 00000100:00100000:2.0:1787784646.395697:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:20527:x1874627657970816:12345-192.168.203.43@tcp:101:cp.0 Request processed in 6790us (7152us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.395707:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 166 00000100:00100000:1.0:1787784646.407850:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.407958:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657971200 00000100:00100000:3.0:1787784646.408217:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668331904:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784646.408301:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657971200 00000100:00100000:0.0:1787784646.408502:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 133 00000100:00100000:0.0:1787784646.408515:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+32:20527:x1874627657971200:12345-192.168.203.43@tcp:35:cp.0 00010000:00010000:3.0:1787784646.408696:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784646.408878:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657970944:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 15172us (15781us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784646.408890:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 185 00000100:00100000:1.0:1787784646.409691:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+33:20527:x1874627657971200:12345-192.168.203.43@tcp:35:cp.0 Request processed in 1038us (1717us total) trans 4294967724 rc 0/0 00000100:00100000:1.0:1787784646.409707:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 133 00000100:00100000:3.0:1787784646.453442:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.453714:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657971328 00000100:00100000:2.0:1787784646.453729:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657971328 00000100:00100000:2.0:1787784646.453806:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 646 00000100:00100000:2.0:1787784646.453819:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+67:20530:x1874627657971328:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784646.453869:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.453897:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc462a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07b6 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.453988:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454006:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454019:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454037:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454045:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454052:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454239:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc463f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454249:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc463f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454256:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc463f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784646.454313:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x11:0x0]: returning size 158 00000004:00200000:2.0:1787784646.454325:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784646.454352:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc463f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454366:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc463f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454377:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454384:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454391:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454397:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.454402:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454410:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454425:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:2.0:1787784646.454429:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.454434:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454441:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4631 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454454:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454471:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454477:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454482:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784646.454485:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454492:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454501:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff989530cb1500) 00010000:00010000:2.0:1787784646.454504:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784646.454506:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454512:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4638 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.454550:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc462a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07b6 expref: 69 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.455056:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb800/0xb0ce63137acc463f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07b6 expref: 68 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.455066:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:2.0:1787784646.455068:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784646.455080:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb800, rc 0) 00000100:00100000:2.0:1787784646.455657:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:20530:x1874627657971328:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1837us (2228us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.455668:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 646 00000100:00100000:0.0:1787784646.459299:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.459457:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657971456 00000100:00100000:2.0:1787784646.459470:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657971456 00000100:00100000:2.0:1787784646.459557:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 167 00000100:00100000:2.0:1787784646.459577:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657971456:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784646.459601:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784646.459624:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4646 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07bd expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.459703:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668332032:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784646.459801:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180180 going to sleep for 11 seconds 00000100:00080000:1.0:1787784646.460665:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1787784646.463137:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784646.463195:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime: 0 -> 1787784645 00002000:00010000:2.0:1787784646.463198:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb atime: 0 -> 1787784645 00002000:00010000:2.0:1787784646.463200:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime: 0 -> 1787784645 00000100:00100000:2.0:1787784646.463233:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668332032:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784646.463293:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4646 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07bd expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.463323:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4646 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07bd expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.463340:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb600, rc 0) 00000100:00100000:2.0:1787784646.463431:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657971456:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3854us (4152us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784646.463442:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 167 00000100:00100000:0.0:1787784646.466930:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784646.467031:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657971584 00000100:00100000:1.0:1787784646.467043:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657971584 00000100:00100000:1.0:1787784646.467111:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 647 00000100:00100000:1.0:1787784646.467121:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:20530:x1874627657971584:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784646.467181:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784646.467208:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc464d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07c4 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.467299:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc4654 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.467314:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc4654 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.467327:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc4654 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784646.467402:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 512 00000004:00200000:1.0:1787784646.467406:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784646.467430:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc4654 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.467460:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc464d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07c4 expref: 70 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.467498:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134400/0xb0ce63137acc4654 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07c4 expref: 69 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.467507:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff989519d44d00) 00010000:00010000:1.0:1787784646.467511:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784646.467514:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539134400, rc 0) 00000100:00100000:1.0:1787784646.467602:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:20530:x1874627657971584:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 482us (687us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784646.467609:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 647 00000100:00100000:1.0:1787784646.473849:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.474962:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657971712 00000100:00100000:3.0:1787784646.474977:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657971712 00000100:00100000:3.0:1787784646.475065:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 168 00000100:00100000:3.0:1787784646.475077:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2356:x1874627657971712:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784646.475102:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784646.475127:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000668d09cf/0xb0ce63137acc465b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07cb expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784646.475257:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539103480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668332160:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784646.475607:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895012549c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784646.479054:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539103480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668332160:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784646.479111:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000668d09cf/0xb0ce63137acc465b lrc: 1/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07cb expref: 10 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.479428:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000668d09cf/0xb0ce63137acc465b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07cb expref: 10 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.479445:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3000, rc 0) 00000100:00100000:3.0:1787784646.479572:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2356:x1874627657971712:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4485us (5725us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784646.479583:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 168 00000100:00100000:1.0:1787784646.484248:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.484390:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657971968 00000100:00100000:3.0:1787784646.484403:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657971968 00000100:00100000:3.0:1787784646.484480:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 648 00000100:00100000:3.0:1787784646.484492:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:20530:x1874627657971968:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784646.485452:0:5811:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].15, 136 off10232 00000100:00100000:3.0:1787784646.485655:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:20530:x1874627657971968:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 1161us (1418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784646.485667:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 648 00000040:00080000:0.0:1787784646.513547:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784646.513576:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784646.513624:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 16 count 3 00000040:00080000:0.0:1787784646.513632:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784646.513995:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304bc00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668332288:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784646.514123:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5480 going to sleep for 11 seconds 00000100:00100000:0.0:1787784646.520897:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304bc00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668332288:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784646.527379:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.527654:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657972096 00000100:00100000:2.0:1787784646.527669:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657972096 00000100:00100000:2.0:1787784646.527902:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00100000:2.0:1787784646.527916:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+33:20536:x1874627657972096:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784646.527968:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x11:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784646.528224:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+33:20536:x1874627657972096:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 311us (860us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.528234:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00100000:1.0:1787784646.543875:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.544767:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657972352 00000100:00100000:0.0:1787784646.544782:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657972352 00000100:00100000:0.0:1787784646.544857:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 650 00000100:00100000:0.0:1787784646.544869:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+33:20536:x1874627657972352:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784646.544891:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784646.544913:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4662 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07d2 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.545021:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4669 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.545055:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4669 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.545068:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4669 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784646.545081:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x11:0x0]: returning size 158 00010000:00010000:0.0:1787784646.545234:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4662 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07d2 expref: 36 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.545249:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4669 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07d2 expref: 35 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.545255:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:0.0:1787784646.545257:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784646.545260:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd400, rc 0) 00000100:00100000:0.0:1787784646.545324:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:20536:x1874627657972352:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 458us (1464us total) trans 4294967725 rc 0/0 00000100:00100000:0.0:1787784646.545331:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 650 00000100:00100000:1.0:1787784646.548924:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.549108:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657972480 00000100:00100000:0.0:1787784646.549121:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657972480 00000100:00100000:0.0:1787784646.549226:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 25 00000100:00100000:0.0:1787784646.549237:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:2356:x1874627657972480:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784646.549255:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784646.549280:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4669 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07d2 expref: 35 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.549334:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:0.0:1787784646.549339:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784646.549398:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2356:x1874627657972480:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 163us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.549406:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 25 00000100:00100000:1.0:1787784646.550319:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784646.550503:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657972608 00000100:00100000:2.0:1787784646.550513:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657972608 00000100:00100000:2.0:1787784646.550581:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 169 00000100:00100000:2.0:1787784646.550592:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657972608:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784646.550649:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.550674:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 9 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.550685:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784646.550691:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784646.550696:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 4/1,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.550742:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 9 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.550773:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 9 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.550792:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668332416:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784646.550886:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784646.555003:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b112c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668332416:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784646.555044:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784646.556636:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.556833:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657972736 00000100:00100000:3.0:1787784646.556845:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657972736 00002000:00100000:3.0:1787784646.556911:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xf:0x0] (0->157) req@ffff9894018343c0 x1874627657972736/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:527/0 lens 648/0 e 0 to 0 dl 1787784657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784646.556926:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894018343c0 with x1874627657972736 ext(0->157) 00010000:00010000:3.0:1787784646.556937:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 10 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.556978:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 10 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784646.556987:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9894018343c0 00000100:00100000:3.0:1787784646.557238:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 5 00000100:00100000:3.0:1787784646.557251:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018343c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657972736:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:3.0:1787784646.557958:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018343c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627657972736:12345-192.168.203.43@tcp:4:cp.0 Request processed in 704us (1330us total) trans 4294967330 rc 0/0 00000100:00100000:3.0:1787784646.557970:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 5 00002000:00100000:3.0:1787784646.557980:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xf:0x0] (0->157) req@ffff9894018343c0 x1874627657972736/t4294967330(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:527/0 lens 648/448 e 0 to 0 dl 1787784657 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784646.557990:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894018343c0 with x1874627657972736 ext(0->157) 00010000:00010000:3.0:1787784646.557999:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 12 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.558018:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 12 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784646.558027:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9894018343c0 00000100:00100000:1.0:1787784646.560716:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.560798:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657972864 00000100:00100000:0.0:1787784646.560811:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657972864 00010000:00010000:0.0:1787784646.560861:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 12 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784646.560905:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 186 00000100:00100000:0.0:1787784646.560916:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627657972864:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784646.560936:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784646.560963:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 12 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.560978:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 12 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.561006:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4623 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07af expref: 12 pid: 10688 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.561027:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784646.561032:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784646.561039:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.561069:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.561109:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784646.561441:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657972864:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 524us (738us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.561451:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 186 00010000:00010000:2.0:1787784646.561535:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561547:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561592:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561602:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561611:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561632:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784646.561637:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561647:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561660:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784646.561668:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005991176b/0xb0ce63137acc4677 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784646.561726:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657972608:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 11136us (11418us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784646.561735:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 169 00000100:00100000:1.0:1787784646.564726:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.564785:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657973120 00000100:00100000:0.0:1787784646.564798:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657973120 00000100:00100000:0.0:1787784646.564874:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 651 00000100:00100000:0.0:1787784646.564886:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:20536:x1874627657973120:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784646.565028:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:20536:x1874627657973120:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 144us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.565037:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 651 00000100:00100000:1.0:1787784646.575244:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.575407:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657973248 00000100:00100000:3.0:1787784646.575421:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657973248 00000100:00100000:3.0:1787784646.575494:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 170 00000100:00100000:3.0:1787784646.575508:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657973248:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784646.575547:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784646.575592:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000668d09cf/0xb0ce63137acc467e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07d9 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784646.575629:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 158, mtime: 0x6a8f6dc5, atime: 0x0, ctime: 0x6a8f6dc5, blocks: 0x1 00010000:00010000:3.0:1787784646.575657:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=437s v=(0 0) 00010000:00010000:3.0:1787784646.575663:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784646.575675:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000668d09cf/0xb0ce63137acc467e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07d9 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.575696:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000668d09cf/0xb0ce63137acc467e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07d9 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.575713:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3000, rc 0) 00000100:00100000:3.0:1787784646.575793:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657973248:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 287us (565us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784646.575801:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 170 00010000:00010000:3.0:1787784646.581115:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784646.581445:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.581529:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657973504 00000100:00100000:3.0:1787784646.581541:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657973504 00000100:00100000:3.0:1787784646.581772:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 652 00000100:00100000:3.0:1787784646.581784:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:20536:x1874627657973504:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784646.581931:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:20536:x1874627657973504:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 148us (493us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784646.581940:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 652 00000100:00100000:1.0:1787784646.585375:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.585938:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657973632 00000100:00100000:3.0:1787784646.585953:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657973632 00002000:00100000:3.0:1787784646.586020:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xf:0x0] (0->4095) req@ffff9894018370c0 x1874627657973632/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:527/0 lens 488/0 e 0 to 0 dl 1787784657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784646.586033:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894018370c0 with x1874627657973632 ext(0->4095) 00002000:00010000:3.0:1787784646.586040:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894018370c0 00000100:00100000:3.0:1787784646.586135:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 55 00000100:00100000:3.0:1787784646.586179:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018370c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657973632:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784646.586540:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018370c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657973632:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 393us (1179us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784646.586549:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 55 00002000:00100000:3.0:1787784646.586571:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xf:0x0] (0->4095) req@ffff9894018370c0 x1874627657973632/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:527/0 lens 488/4536 e 0 to 0 dl 1787784657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784646.586580:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894018370c0 with x1874627657973632 ext(0->4095) 00002000:00010000:3.0:1787784646.586593:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894018370c0 00000100:00100000:1.0:1787784646.591510:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.591767:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657973760 00000100:00100000:0.0:1787784646.591781:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657973760 00000100:00100000:0.0:1787784646.591901:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 653 00000100:00100000:0.0:1787784646.591914:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:20536:x1874627657973760:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784646.591942:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784646.591969:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4685 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07e0 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.592025:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.592062:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.592076:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.592233:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4685 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 36 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.592256:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 35 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.592266:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:0.0:1787784646.592269:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784646.592273:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784646.592364:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:20536:x1874627657973760:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 453us (870us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.592374:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 653 00000100:00100000:1.0:1787784646.597249:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.597396:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657973888 00000100:00100000:0.0:1787784646.597409:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657973888 00000100:00100000:0.0:1787784646.597473:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 654 00000100:00100000:0.0:1787784646.597484:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:20536:x1874627657973888:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784646.597525:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x11:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784646.597863:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:20536:x1874627657973888:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 379us (626us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.597874:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 654 00000100:00100000:3.0:1787784646.601838:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784646.601970:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657974016 00000100:00100000:0.0:1787784646.601983:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657974016 00000100:00100000:0.0:1787784646.602052:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 171 00000100:00100000:0.0:1787784646.602064:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657974016:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784646.602125:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602184:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784646.602190:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784646.602197:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602214:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602255:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602265:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602274:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602291:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784646.602296:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602320:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602340:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.602349:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007c524efc/0xb0ce63137acc4693 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784646.602443:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657974016:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 380us (619us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.602452:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 171 00000100:00100000:3.0:1787784646.604852:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784646.605052:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657974272 00000100:00100000:1.0:1787784646.605064:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657974272 00000100:00100000:1.0:1787784646.605137:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 655 00000100:00100000:1.0:1787784646.605183:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:20536:x1874627657974272:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784646.605301:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.605321:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 35 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.605331:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 4/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.605376:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 35 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784646.605392:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 35 pid: 10850 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784646.605407:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508851e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668332544:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784646.605536:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784646.609346:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.609536:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508851e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668332544:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784646.609586:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784646.609723:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657974400 00000100:00100000:0.0:1787784646.609732:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657974400 00010000:00010000:0.0:1787784646.609781:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 36 pid: 10850 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784646.609828:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:0.0:1787784646.609839:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:2356:x1874627657974400:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784646.609859:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784646.609867:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 36 pid: 10850 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.609881:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 36 pid: 10850 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.609927:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc468c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e0 expref: 36 pid: 10850 timeout: 537 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.609945:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:0.0:1787784646.609950:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.609960:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.609967:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784646.610001:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784646.610069:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+35:2356:x1874627657974400:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 232us (735us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784646.610078:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 187 00010000:00010000:3.0:1787784646.612757:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.612779:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.613079:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.613089:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.613097:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.613105:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784646.613109:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.613118:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.613139:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:3.0:1787784646.613389:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784646.613398:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784646.613407:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc469a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784646.613739:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:20536:x1874627657974272:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 8552us (8894us total) trans 4294967726 rc 0/0 00000100:00100000:3.0:1787784646.613751:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 655 00000040:00080000:0.0:1787784646.615867:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784646.615879:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784646.617136:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784646.619840:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784646.620517:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784646.623184:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657974656 00000100:00100000:3.0:1787784646.623206:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657974656 00000100:00100000:3.0:1787784646.623295:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 134 00000100:00100000:3.0:1787784646.623327:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+35:20536:x1874627657974656:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784646.623687:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:20536:x1874627657974656:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 362us (3178us total) trans 4294967727 rc 0/0 00000100:00100000:3.0:1787784646.623696:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 134 00000040:00080000:3.0:1787784647.121125:0:16651:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784647.121139:0:16651:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784647.121215:0:16651:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 16 count 3 00000040:00080000:3.0:1787784647.121222:0:16651:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784647.194321:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784647.194496:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657974912 00000100:00100000:0.0:1787784647.194513:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657974912 00000100:00100000:0.0:1787784647.194598:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 656 00000100:00100000:0.0:1787784647.194611:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:20543:x1874627657974912:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784647.194639:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784647.194668:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc46a1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07e7 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.194826:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc46a8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.194844:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc46a8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.194859:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc46a8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784647.194873:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x11:0x0]: returning size 158 00010000:00010000:0.0:1787784647.195051:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc46a1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e7 expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.195070:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc46a8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e7 expref: 71 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.195080:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:0.0:1787784647.195083:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784647.195087:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbda00, rc 0) 00000100:00100000:0.0:1787784647.203760:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:20543:x1874627657974912:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 9131us (9429us total) trans 4294967728 rc 0/0 00000100:00100000:0.0:1787784647.203780:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 656 00000100:00100000:0.0:1787784647.211967:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784647.212115:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784647.212193:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657975168 00000100:00100000:0.0:1787784647.212205:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657975168 00000100:00100000:0.0:1787784647.212290:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 657 00000100:00100000:0.0:1787784647.212302:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:20543:x1874627657975168:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:0.0:1787784647.212425:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212460:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212475:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212510:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212518:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212526:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212534:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784647.212539:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212548:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212574:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:0.0:1787784647.212578:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784647.212584:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784647.212592:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc46b6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784647.212703:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:20543:x1874627657975168:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 403us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784647.212712:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 657 00000100:00100000:2.0:1787784647.214644:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657975040 00000100:00100000:2.0:1787784647.214663:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657975040 00000100:00100000:2.0:1787784647.214756:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 26 00000100:00100000:2.0:1787784647.214767:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:2355:x1874627657975040:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784647.214785:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784647.214820:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc46a8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07e7 expref: 71 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784647.214843:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff989530cb1b00) 00010000:00010000:2.0:1787784647.214861:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784647.214945:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+70:2355:x1874627657975040:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 180us (6824us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784647.214955:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 26 00000100:00100000:3.0:1787784647.219923:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784647.221831:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657975424 00000100:00100000:0.0:1787784647.221851:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657975424 00000100:00100000:1.0:1787784647.221990:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 135 00000100:00100000:1.0:1787784647.222004:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+70:20543:x1874627657975424:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:1.0:1787784647.222388:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:20543:x1874627657975424:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 384us (2477us total) trans 4294967729 rc 0/0 00000100:00100000:1.0:1787784647.222398:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 135 00010000:00010000:0.0:1787784647.602304:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784647.724670:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967330 is committed 00000001:00080000:1.0:1787784647.750961:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967728 is committed 00000001:00080000:1.0:1787784647.750981:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967725 is committed 00000001:00080000:1.0:1787784647.750991:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967729 is committed 00000001:00080000:1.0:1787784647.751005:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967727 is committed 00010000:00010000:3.0:1787784648.626251:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784649.203010:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668332672:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784649.203119:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784649.203257:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668332672 00000100:00100000:3.0:1787784649.203265:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668332672 00000100:00100000:3.0:1787784649.203326:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:3.0:1787784649.203336:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668332672:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784649.203384:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784649.203431:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668332672:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784649.203440:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:3.0:1787784649.203486:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668332672:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784649.203596:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784649.479266:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784649.479307:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784649.479405:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784649.479408:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657975552 00000100:00100000:1.0:1787784649.479417:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657975552 00000100:00100000:3.0:1787784649.479426:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784649.479443:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784649.479460:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784649.479466:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 343 00000100:00100000:1.0:1787784649.479475:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657975552:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784649.479479:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784649.479508:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657976064 00000100:00100000:3.0:1787784649.479511:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657976064 00000100:00100000:1.0:1787784649.479553:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657975552:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 66us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784649.479562:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 343 00000100:00100000:3.0:1787784649.479612:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657975680 00000100:00100000:3.0:1787784649.479616:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657975680 00000100:00100000:3.0:1787784649.479695:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:3.0:1787784649.479703:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2354:x1874627657976064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784649.479752:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2354:x1874627657976064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 50us (347us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784649.479758:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:3.0:1787784649.479772:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:3.0:1787784649.479778:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:2354:x1874627657975680:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784649.479803:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:2354:x1874627657975680:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784649.479810:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:0.0:1787784649.479871:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657975808 00000100:00100000:0.0:1787784649.479876:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657975808 00000100:00100000:0.0:1787784649.479907:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657976320 00000100:00100000:0.0:1787784649.479910:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657976320 00000100:00100000:0.0:1787784649.479922:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657975936 00000100:00100000:0.0:1787784649.479925:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657975936 00000100:00100000:0.0:1787784649.479934:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657976192 00000100:00100000:0.0:1787784649.479936:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657976192 00000100:00100000:0.0:1787784649.479948:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 204 00000100:00100000:0.0:1787784649.479954:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657975808:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784649.479983:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657975808:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (675us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784649.479989:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 204 00000100:00100000:0.0:1787784649.480009:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 205 00000100:00100000:0.0:1787784649.480015:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627657976320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784649.480039:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627657976320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (612us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784649.480056:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 205 00000100:00100000:0.0:1787784649.480069:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 206 00000100:00100000:0.0:1787784649.480075:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657975936:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784649.480098:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657975936:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (637us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784649.480104:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 206 00000100:00100000:0.0:1787784649.480117:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 207 00000100:00100000:0.0:1787784649.480123:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657976192:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784649.480142:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657976192:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (662us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784649.480165:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 207 00000100:00080000:3.0:1787784649.483245:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784649.483255:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784649.483258:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784649.650018:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784650.676521:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784651.250830:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668333184:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.250903:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784651.250975:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1787784651.250982:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668332800:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.251011:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784651.251037:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668333312:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.251115:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784651.251144:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668332928:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.251203:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784651.251217:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668333440:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.251236:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784651.251253:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1787784651.251347:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668333184 00000100:00100000:0.0:1787784651.251355:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668333184 00000100:00100000:0.0:1787784651.251398:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668333440 00000100:00100000:0.0:1787784651.251401:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668333440 00000100:00100000:0.0:1787784651.251420:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1787784651.251429:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668333184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784651.251478:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668333184:12345-0@lo:400:kworker.0 Request processed in 51us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784651.251485:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1787784651.251506:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:0.0:1787784651.251512:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668333440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784651.251548:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668333440:12345-0@lo:400:kworker.0 Request processed in 36us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784651.251567:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:0.0:1787784651.251608:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668333312 00000100:00100000:0.0:1787784651.251668:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668333312 00000100:00100000:0.0:1787784651.251695:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668332928 00000100:00100000:0.0:1787784651.251698:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668332928 00000100:00100000:0.0:1787784651.251713:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:0.0:1787784651.251719:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668333312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784651.251760:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668333312:12345-0@lo:400:kworker.0 Request processed in 41us (645us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784651.251766:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:0.0:1787784651.251778:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:0.0:1787784651.251784:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668332928:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784651.251807:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668332928:12345-0@lo:400:kworker.0 Request processed in 23us (604us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784651.251812:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:0.0:1787784651.251871:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668333312:0@lo:400:kworker.0 00000100:00100000:0.0:1787784651.251883:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668333184:0@lo:400:kworker.0 00000100:00100000:3.0:1787784651.252103:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668332800 00000100:00100000:3.0:1787784651.252108:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668332800 00000100:00100000:3.0:1787784651.252144:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:3.0:1787784651.252175:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668332800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784651.252214:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668332800:12345-0@lo:400:kworker.0 Request processed in 41us (1202us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784651.252221:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00080000:3.0:1787784651.252249:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784651.252254:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784651.252274:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784651.252304:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668332800:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.252463:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668333056:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.252496:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784651.252534:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668333568:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784651.252559:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784651.252584:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668332928:0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.252593:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668333440:0@lo:400:kworker.0 00000100:00100000:3.0:1787784651.252686:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668333056 00000100:00100000:3.0:1787784651.252691:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668333056 00000100:00100000:3.0:1787784651.252727:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1787784651.252793:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668333056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784651.252917:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668333568 00000100:00100000:2.0:1787784651.252922:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668333568 00000100:00100000:2.0:1787784651.252959:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:2.0:1787784651.252965:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668333568:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784651.253369:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3857711104 avail 3855613952 00000100:00080000:0.0:1787784651.253443:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784651.257334:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668333056:12345-0@lo:400:kworker.0 Request processed in 4591us (4827us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784651.257349:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1787784651.259978:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668333568:12345-0@lo:13:osp-pre-1-0.0 Request processed in 7004us (7409us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784651.262203:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:0.0:1787784651.264218:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668333056:0@lo:400:kworker.0 00000100:00100000:0.0:1787784651.264323:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668333568:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1787784651.698495:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784651.762066:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784651.762076:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784651.762879:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 16 count 2 00000040:00080000:0.0:1787784651.762889:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784652.520489:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784652.520544:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657976448 00000100:00100000:3.0:1787784652.520553:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657976448 00000100:00100000:3.0:1787784652.520616:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 658 00000100:00100000:3.0:1787784652.520627:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:20523:x1874627657976448:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1787784652.520723:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:20523:x1874627657976448:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 98us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784652.520731:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 658 00000100:00080000:3.0:1787784652.520753:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784652.522692:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784652.522728:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657976704 00000100:00100000:1.0:1787784652.522737:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657976704 00000100:00100000:1.0:1787784652.522789:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 136 00000100:00100000:1.0:1787784652.522799:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:20523:x1874627657976704:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784652.525624:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:20523:x1874627657976704:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 2816us (2929us total) trans 4294967730 rc 0/0 00000100:00100000:1.0:1787784652.525638:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 136 00010000:00010000:3.0:1787784652.721603:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787784652.872822:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967730 is committed 00010000:00010000:1.0:1787784653.748354:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787784654.124808:0:16899:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 11a: Import a file ==================== 18:50:52 (1787784652) 00000100:00100000:3.0:1787784654.248052:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.253409:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657976832 00000100:00100000:0.0:1787784654.253421:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657976832 00000100:00100000:0.0:1787784654.253517:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 659 00000100:00100000:0.0:1787784654.253529:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:21124:x1874627657976832:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784654.253548:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784654.253602:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46bd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07ee expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253680:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253694:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253707:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253731:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253738:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253745:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253972:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253980:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253993:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.253999:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784654.254003:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254010:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254021:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:0.0:1787784654.254025:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784654.254029:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254035:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc46c4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254044:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254050:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254055:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254060:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbde00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784654.254062:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254076:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254084:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff989503a6da00) 00010000:00010000:0.0:1787784654.254086:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784654.254089:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254095:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46cb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254122:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46bd lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07ee expref: 69 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254133:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:0.0:1787784654.254136:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784654.254139:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46bd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07ee expref: 69 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.254167:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784654.254228:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:21124:x1874627657976832:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 701us (6184us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784654.254236:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 659 00000100:00080000:1.0:1787784654.255547:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784654.279240:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.279277:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657977088 00000100:00100000:0.0:1787784654.279300:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657977088 00000100:00100000:0.0:1787784654.279351:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 660 00000100:00100000:0.0:1787784654.279360:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:21124:x1874627657977088:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784654.279421:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784654.279462:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4543 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0762 expref: 68 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.279480:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:0.0:1787784654.279485:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784654.279809:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.279823:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 28 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.279833:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.279862:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 28 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.279875:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 28 pid: 5812 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.279887:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668333696:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784654.279975:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784654.289991:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668333696:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784654.290020:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.290619:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784654.290702:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657977216 00000100:00100000:1.0:1787784654.290710:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657977216 00010000:00010000:1.0:1787784654.290763:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 29 pid: 5812 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784654.290802:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 188 00000100:00100000:1.0:1787784654.290819:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2356:x1874627657977216:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784654.290834:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784654.290841:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 29 pid: 5812 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.290855:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 29 pid: 5812 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.290876:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4582 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0777 expref: 29 pid: 5812 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.290890:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:1.0:1787784654.290895:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.290904:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.290911:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.290931:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784654.290981:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+28:2356:x1874627657977216:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 164us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784654.290990:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 188 00010000:00010000:0.0:1787784654.291105:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.291114:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.291166:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.291177:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.291187:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784654.292388:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 512 00010000:00010000:0.0:1787784654.292402:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.292411:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.292418:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.292425:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784654.292428:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.292436:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.292448:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989519d44600) 00010000:00010000:0.0:1787784654.292457:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784654.292463:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.292469:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc46d2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784654.292489:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000381bddb4 reply state 0000000076052964 transno 4294967731 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.292497:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784654.292587:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:21124:x1874627657977088:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 13225us (13354us total) trans 4294967731 rc 0/0 00000100:00100000:0.0:1787784654.292596:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 660 00000100:00080000:3.0:1787784654.292693:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784654.302289:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.302307:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.302329:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.302336:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbde00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784654.302341:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.302348:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.302365:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff989503a6da00) 00010000:00010000:1.0:1787784654.302369:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784654.302373:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.302379:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc46d9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784654.321455:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784654.321481:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668333824:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784654.321562:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784654.321677:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668333824 00000100:00100000:3.0:1787784654.321686:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668333824 00000100:00100000:3.0:1787784654.321759:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:3.0:1787784654.321769:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668333824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784654.321804:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784654.321860:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668333824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (297us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784654.321881:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:3.0:1787784654.321927:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668333824:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787784654.322040:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784654.571950:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784654.572102:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657977344 00000100:00100000:2.0:1787784654.572117:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657977344 00000100:00100000:2.0:1787784654.572247:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 661 00000100:00100000:2.0:1787784654.572258:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:21170:x1874627657977344:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:2.0:1787784654.572284:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784654.572308:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc46e0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07f5 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.572377:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc46e7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.572391:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc46e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.572402:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc46e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784654.572462:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784654.572466:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784654.572490:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc46e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.572525:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc46e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07f5 expref: 70 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.572560:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc46e7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07f5 expref: 69 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.572569:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784654.572572:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784654.572575:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb600, rc 0) 00000100:00100000:2.0:1787784654.572656:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:21170:x1874627657977344:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 400us (720us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784654.572664:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 661 00000100:00100000:0.0:1787784654.576357:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784654.576441:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657977600 00000100:00100000:3.0:1787784654.576455:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657977600 00000100:00100000:3.0:1787784654.576619:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 662 00000100:00100000:3.0:1787784654.576633:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:21170:x1874627657977600:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:3.0:1787784654.576726:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784654.576764:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc46e7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07f5 expref: 69 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.576790:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1500) 00010000:00010000:3.0:1787784654.576803:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784654.577027:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:21170:x1874627657977600:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 397us (683us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787784654.577035:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 662 00000100:00100000:1.0:1787784654.579320:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784654.579436:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657977728 00000100:00100000:3.0:1787784654.579448:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657977728 00000100:00100000:3.0:1787784654.579641:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:3.0:1787784654.579654:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+68:21170:x1874627657977728:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:3.0:1787784654.579691:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784654.579721:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc46ee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b07fc expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.579798:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc46f5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.579837:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc46f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.579851:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc46f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784654.579929:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784654.579933:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784654.579960:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc46f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.580000:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc46ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07fc expref: 70 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.580021:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc46f5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07fc expref: 69 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.580030:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784654.580033:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784654.580036:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3400, rc 0) 00000100:00100000:3.0:1787784654.580123:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:21170:x1874627657977728:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 472us (826us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784654.580132:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:1.0:1787784654.583318:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.583487:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657977856 00000100:00100000:0.0:1787784654.583500:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657977856 00000100:00100000:3.0:1787784654.583667:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 664 00000100:00100000:3.0:1787784654.583957:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+69:21170:x1874627657977856:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:3.0:1787784654.583985:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784654.584013:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc46fc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0803 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.584214:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.584231:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.584244:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.584260:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.584268:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.584275:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591391:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4711 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591416:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4711 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591430:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4711 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784654.591512:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 512 00000004:00200000:2.0:1787784654.591517:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784654.591543:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4711 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591558:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591564:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591589:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591594:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784654.591599:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591607:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591618:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:2.0:1787784654.591621:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784654.591626:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591633:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4703 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591650:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591656:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591662:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591679:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784654.591682:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591700:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591709:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff989505542200) 00010000:00010000:2.0:1787784654.591711:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784654.591715:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591721:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc470a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591772:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc46fc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0803 expref: 71 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591789:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4711 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0803 expref: 70 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.591796:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784654.591798:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784654.591800:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eac00, rc 0) 00000100:00100000:2.0:1787784654.591871:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+70:21170:x1874627657977856:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 7919us (8568us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784654.591880:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 664 00000100:00100000:1.0:1787784654.599263:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784654.599298:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784654.599317:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784654.599338:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784654.599374:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784654.599594:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657977984 00000100:00100000:2.0:1787784654.599604:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657977984 00000100:00100000:2.0:1787784654.599648:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 345 00000100:00100000:2.0:1787784654.599658:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657977984:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784654.599743:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627657977984:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 86us (423us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784654.599751:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 345 00000100:00100000:0.0:1787784654.600813:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657978112 00000100:00100000:0.0:1787784654.600823:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657978112 00000100:00100000:0.0:1787784654.600885:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657978368 00000100:00100000:0.0:1787784654.600888:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657978368 00000100:00100000:0.0:1787784654.600902:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657978240 00000100:00100000:0.0:1787784654.600905:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657978240 00000100:00100000:0.0:1787784654.600917:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657978496 00000100:00100000:0.0:1787784654.600919:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657978496 00000100:00100000:0.0:1787784654.600946:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:0.0:1787784654.600955:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657978112:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784654.601008:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627657978112:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 55us (1755us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784654.601015:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:0.0:1787784654.601050:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 211 00000100:00100000:0.0:1787784654.601057:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657978368:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784654.601086:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627657978368:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (1788us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784654.601092:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 211 00000100:00100000:0.0:1787784654.601117:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 212 00000100:00100000:0.0:1787784654.601124:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657978240:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784654.601168:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2354:x1874627657978240:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (1807us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784654.601174:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 212 00000100:00100000:0.0:1787784654.601187:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 213 00000100:00100000:0.0:1787784654.601192:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627657978496:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784654.601211:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627657978496:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (1836us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784654.601216:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 213 00000100:00080000:1.0:1787784654.601702:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784654.601707:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784654.638982:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.639284:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657978624 00000100:00100000:0.0:1787784654.639293:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657978624 00000100:00100000:0.0:1787784654.639346:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 665 00000100:00100000:0.0:1787784654.639356:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+28:21176:x1874627657978624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784654.639380:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784654.639399:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4718 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b080a expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.639470:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.639496:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.639534:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784654.639594:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784654.639598:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784654.639622:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.639644:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4718 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 30 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.639663:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 29 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.639672:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:0.0:1787784654.639675:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784654.639679:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbdc00, rc 0) 00000100:00100000:0.0:1787784654.639985:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:21176:x1874627657978624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 629us (1011us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784654.639993:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 665 00000100:00100000:0.0:1787784654.658013:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784654.658121:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657978880 00000100:00100000:2.0:1787784654.658130:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657978880 00000100:00100000:2.0:1787784654.658215:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 666 00000100:00100000:2.0:1787784654.658226:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+70:21176:x1874627657978880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784654.658247:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784654.658251:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787784654.658278:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc4711 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0803 expref: 70 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.658298:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784654.658309:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784654.658324:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4726 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0811 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.658456:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.658469:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.658479:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.658539:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.658565:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.658572:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659198:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659209:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659216:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784654.659227:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 0 00010000:00010000:2.0:1787784654.659252:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659259:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659265:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659272:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eaa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784654.659276:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659283:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659304:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff989530cb1600) 00010000:00010000:2.0:1787784654.659307:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784654.659311:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659318:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc472d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784654.659330:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d824ff84 reply state 00000000bca10e91 transno 4294967732 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784654.659338:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784654.659366:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4726 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659382:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 71 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.659390:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1900) 00010000:00010000:2.0:1787784654.659392:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784654.659395:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eba00, rc 0) 00000100:00100000:2.0:1787784654.659468:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+72:21176:x1874627657978880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1244us (1466us total) trans 4294967732 rc 0/0 00000100:00100000:2.0:1787784654.659477:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 666 00010000:00010000:0.0:1787784654.663310:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.663327:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.663353:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.663360:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784654.663364:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.663372:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.663388:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x4a4b4929 (ffff989530cb1c00) 00010000:00010000:0.0:1787784654.663392:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784654.663396:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.663403:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4734 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.663959:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784654.664489:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657979136 00000100:00100000:3.0:1787784654.664499:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657979136 00000100:00100000:3.0:1787784654.664581:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 667 00000100:00100000:3.0:1787784654.664591:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:21176:x1874627657979136:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784654.664622:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784654.664641:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4749 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0818 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.664864:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.664888:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.664897:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.664929:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.664942:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 72 pid: 5813 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784654.664956:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668333952:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784654.665042:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784654.668245:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.668313:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657979264 00000100:00100000:0.0:1787784654.668326:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657979264 00010000:00010000:0.0:1787784654.668364:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 73 pid: 5813 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.668401:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:0.0:1787784654.668411:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057eec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:2356:x1874627657979264:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784654.668423:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784654.668428:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 73 pid: 5813 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784654.668440:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668333952:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784654.668461:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 73 pid: 5813 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.668481:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc473b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0811 expref: 73 pid: 5813 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.668487:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.668493:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1900) 00010000:00010000:0.0:1787784654.668501:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.668528:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.668537:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.669995:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784654.670053:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.670088:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057eec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+72:2356:x1874627657979264:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1672us (1854us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784654.670101:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 189 00010000:00010000:3.0:1787784654.671888:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784654.671929:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 0 00010000:00010000:3.0:1787784654.674876:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.678786:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.678796:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.678802:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784654.678809:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.678819:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.678841:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1900) 00010000:00010000:3.0:1787784654.678845:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784654.678850:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.678858:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4750 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.683076:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4749 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0818 expref: 72 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.683110:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784654.683114:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784654.683124:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4749 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0818 expref: 72 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.684605:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2a00, rc 0) 00000100:00100000:3.0:1787784654.684764:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:21176:x1874627657979136:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 20169us (20804us total) trans 4294967733 rc 301/301 00000100:00100000:3.0:1787784654.684778:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 667 00000100:00100000:2.0:1787784654.687291:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784654.689286:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657979520 00000100:00100000:1.0:1787784654.689307:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657979520 00000100:00100000:1.0:1787784654.689395:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 137 00000100:00100000:1.0:1787784654.689410:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950767b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:21176:x1874627657979520:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784654.689940:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950767b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+72:21176:x1874627657979520:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 532us (2663us total) trans 4294967734 rc 0/0 00000100:00100000:1.0:1787784654.689949:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 137 00000100:00100000:1.0:1787784654.696305:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784654.698114:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657979648 00000100:00100000:2.0:1787784654.698139:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657979648 00000100:00100000:2.0:1787784654.698299:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 668 00000100:00100000:2.0:1787784654.698311:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+72:21176:x1874627657979648:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784654.698344:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784654.698376:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc475e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b081f expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.698455:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc475e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b081f expref: 73 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.698474:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eac00/0xb0ce63137acc475e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b081f expref: 73 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784654.698498:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1600) 00010000:00010000:2.0:1787784654.698512:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784654.698517:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eac00, rc 0) 00000100:00100000:2.0:1787784654.698619:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21176:x1874627657979648:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 309us (2328us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784654.698627:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 668 00000100:00100000:2.0:1787784654.703263:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.703434:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657979904 00000100:00100000:0.0:1787784654.703449:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657979904 00000100:00100000:0.0:1787784654.703543:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00100000:0.0:1787784654.703557:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21176:x1874627657979904:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1787784654.703656:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.703680:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.703719:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.704023:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.704032:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.704039:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.704046:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784654.704052:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.704060:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.704558:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1600) 00010000:00010000:0.0:1787784654.704562:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787784654.704569:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.704576:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4765 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.704688:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:21176:x1874627657979904:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 1132us (1437us total) trans 4294967735 rc 0/0 00000100:00100000:0.0:1787784654.704699:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00100000:1.0:1787784654.711533:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.711771:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657980160 00000100:00100000:0.0:1787784654.711787:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657980160 00000100:00100000:0.0:1787784654.711867:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 670 00000100:00100000:0.0:1787784654.711878:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:21176:x1874627657980160:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784654.712476:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.712512:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.712527:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784654.712615:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967736 00000004:00010000:0.0:1787784654.712661:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b160572a reply state 0000000099de7e52 transno 4294967737 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784654.712672:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784654.712876:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:21176:x1874627657980160:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 999us (1371us total) trans 4294967737 rc 0/0 00000100:00100000:0.0:1787784654.712883:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 670 00000020:00080000:0.0:1787784654.712888:0:5813:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1787784654.724961:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.725204:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657980416 00000100:00100000:0.0:1787784654.725216:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657980416 00000100:00100000:0.0:1787784654.725284:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 138 00000100:00100000:0.0:1787784654.725295:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:21176:x1874627657980416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784654.725684:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:21176:x1874627657980416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 389us (737us total) trans 4294967738 rc 0/0 00000100:00100000:0.0:1787784654.725694:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 138 00010000:00010000:3.0:1787784654.728372:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.728396:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.728405:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.728413:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784654.728420:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.728428:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.728452:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1600) 00010000:00010000:3.0:1787784654.728457:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787784654.728463:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.728470:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc476c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.770039:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784654.782527:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.782658:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657980544 00000100:00100000:0.0:1787784654.782685:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657980544 00000100:00100000:0.0:1787784654.782761:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 671 00000100:00100000:0.0:1787784654.782774:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:21179:x1874627657980544:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784654.782809:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784654.782838:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc4773 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0826 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.782927:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.782944:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.782958:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.782976:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.782983:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.782990:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783298:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4788 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783307:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783338:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784654.783382:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 512 00000004:00200000:0.0:1787784654.783386:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784654.783412:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783427:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783434:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783441:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783448:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784654.783453:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783462:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783476:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:0.0:1787784654.783479:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784654.783485:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783492:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc477a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783533:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783540:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783545:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783550:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbca00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784654.783554:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783561:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783569:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff989503a6dc00) 00010000:00010000:0.0:1787784654.783572:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784654.783575:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783580:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc4781 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783619:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc4773 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0826 expref: 79 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783634:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd400/0xb0ce63137acc4788 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0826 expref: 78 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784654.783655:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff989503a6d700) 00010000:00010000:0.0:1787784654.783657:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784654.783661:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd400, rc 0) 00000100:00100000:0.0:1787784654.783740:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:21179:x1874627657980544:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 968us (1240us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784654.783749:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 671 00000100:00100000:2.0:1787784654.788453:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784654.788796:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657980800 00000100:00100000:1.0:1787784654.788810:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657980800 00000100:00100000:1.0:1787784654.788892:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 672 00000100:00100000:1.0:1787784654.788905:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:21179:x1874627657980800:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784654.788934:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784654.788965:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc478f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b082d expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.793841:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.793867:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.793880:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.793916:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.793934:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.793941:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.795116:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc47a4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.795131:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc47a4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796575:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc47a4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784654.796592:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 158 00010000:00010000:1.0:1787784654.796723:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796731:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796737:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796743:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784654.796747:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796773:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796787:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff989503a6d700) 00010000:00010000:1.0:1787784654.796790:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784654.796795:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796801:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc4796 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796817:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796824:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796830:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796835:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784654.796838:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796844:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796852:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x4a4b4929 (ffff9895029d1900) 00010000:00010000:1.0:1787784654.796854:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784654.796858:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796870:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc479d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796931:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc478f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b082d expref: 81 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.796946:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac800/0xb0ce63137acc47a4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b082d expref: 80 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784654.798498:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1600) 00010000:00010000:1.0:1787784654.798511:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784654.798515:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac800, rc 0) 00000100:00100000:1.0:1787784654.798891:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:21179:x1874627657980800:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 9984us (10451us total) trans 4294967739 rc 0/0 00000100:00100000:1.0:1787784654.798902:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 672 00000100:00100000:1.0:1787784654.803549:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784654.803661:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657980928 00000100:00100000:3.0:1787784654.803671:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657980928 00000100:00100000:3.0:1787784654.803728:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00100000:3.0:1787784654.803739:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:21179:x1874627657980928:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:3.0:1787784654.803816:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803831:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803842:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803867:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803873:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803879:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803885:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784654.803888:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803895:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803905:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1600) 00010000:00010000:3.0:1787784654.803909:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784654.803913:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784654.803920:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47b2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784654.804020:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:21179:x1874627657980928:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 282us (479us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784654.804028:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00100000:2.0:1787784654.814026:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784654.814128:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657981184 00000100:00100000:0.0:1787784654.814137:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657981184 00000100:00100000:0.0:1787784654.814219:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 139 00000100:00100000:0.0:1787784654.814229:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:21179:x1874627657981184:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:0.0:1787784654.814908:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:21179:x1874627657981184:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 678us (890us total) trans 4294967740 rc 0/0 00000100:00100000:0.0:1787784654.814917:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 139 00000100:00100000:2.0:1787784655.069831:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784655.070244:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657981440 00000100:00100000:0.0:1787784655.070254:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657981440 00000100:00100000:0.0:1787784655.070376:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 674 00000100:00100000:0.0:1787784655.070388:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:21198:x1874627657981440:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784655.070415:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784655.070437:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc47b9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0834 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784655.070584:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 512 00010000:00010000:0.0:1787784655.070714:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc47b9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0834 expref: 83 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784655.070740:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:0.0:1787784655.070743:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784655.070748:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc47b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0834 expref: 83 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784655.070763:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbda00, rc 0) 00000100:00100000:0.0:1787784655.070844:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:21198:x1874627657981440:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 459us (1023us total) trans 4294967741 rc 301/301 00000100:00100000:0.0:1787784655.070852:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 674 00000100:00100000:1.0:1787784655.077492:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784655.078267:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657981568 00000100:00100000:3.0:1787784655.078284:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657981568 00000100:00100000:3.0:1787784655.078381:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 675 00000100:00100000:3.0:1787784655.078394:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:21198:x1874627657981568:12345-192.168.203.43@tcp:34:lfs.0 00010000:00010000:3.0:1787784655.078727:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078753:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078771:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078791:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078797:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078804:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078944:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078952:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.078981:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784655.079035:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 158 00000004:00200000:3.0:1787784655.079040:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784655.079055:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079071:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079078:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079086:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079092:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784655.079097:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079105:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079119:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff989503a6d700) 00010000:00010000:3.0:1787784655.079123:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784655.079129:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079288:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc47c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079319:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079324:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.079330:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080622:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784655.080640:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080654:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080683:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x4a4b4929 (ffff989505542600) 00010000:00010000:3.0:1787784655.080687:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784655.080691:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 2/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080699:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080729:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080737:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080744:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080769:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784655.080772:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080779:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080788:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff989530cb1600) 00010000:00010000:3.0:1787784655.080791:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784655.080794:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784655.080799:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784655.080945:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:21198:x1874627657981568:12345-192.168.203.43@tcp:34:lfs.0 Request processed in 2547us (3462us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784655.080957:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 675 00000100:00100000:2.0:1787784655.084932:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784655.085037:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657981824 00000100:00100000:1.0:1787784655.085051:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657981824 00000100:00100000:1.0:1787784655.085142:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 140 00000100:00100000:1.0:1787784655.085198:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:21198:x1874627657981824:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:1.0:1787784655.085589:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21198:x1874627657981824:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 394us (668us total) trans 4294967742 rc 0/0 00000100:00100000:1.0:1787784655.085598:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 140 00010000:00010000:1.0:1787784655.793856:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784656.370289:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784656.370313:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668334080:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784656.370375:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784656.370441:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334080 00000100:00100000:0.0:1787784656.370447:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334080 00000100:00100000:1.0:1787784656.370747:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668334208:0@lo:400:kworker.0 00000100:00100000:1.0:1787784656.370785:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784656.370807:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668334720:0@lo:400:kworker.0 00000100:00100000:1.0:1787784656.370827:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784656.370843:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784656.370847:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668334336:0@lo:400:kworker.0 00000100:00100000:1.0:1787784656.370868:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1787784656.370890:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784656.370901:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334720 00000100:00100000:1.0:1787784656.370905:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334720 00000100:00100000:1.0:1787784656.370934:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334336 00000100:00100000:1.0:1787784656.370936:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334336 00000100:00100000:1.0:1787784656.370964:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:1.0:1787784656.370972:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668334720:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784656.371015:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668334720:12345-0@lo:400:kworker.0 Request processed in 44us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784656.371022:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:1.0:1787784656.371048:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:1.0:1787784656.371055:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668334336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371076:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334208 00000100:00100000:3.0:1787784656.371080:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334208 00000100:00100000:1.0:1787784656.371097:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668334336:12345-0@lo:400:kworker.0 Request processed in 42us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784656.371102:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00080000:1.0:1787784656.371120:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784656.371142:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668334336:0@lo:400:kworker.0 00000100:00100000:1.0:1787784656.371181:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668334720:0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371193:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1787784656.371200:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668334208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371240:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668334208:12345-0@lo:400:kworker.0 Request processed in 41us (454us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784656.371245:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1787784656.371381:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668334464:0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371409:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784656.371430:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668334208:0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371434:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784656.371439:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668334592:0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371463:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784656.371486:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334464 00000100:00100000:3.0:1787784656.371499:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334464 00000100:00100000:3.0:1787784656.371529:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:3.0:1787784656.371535:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668334464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371567:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668334464:12345-0@lo:400:kworker.0 Request processed in 32us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784656.371572:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00080000:3.0:1787784656.371585:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784656.371600:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668334464:0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371621:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334592 00000100:00100000:3.0:1787784656.371625:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334592 00000100:00100000:3.0:1787784656.371650:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1787784656.371656:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668334592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784656.371687:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668334592:12345-0@lo:400:kworker.0 Request processed in 31us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784656.371692:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00080000:3.0:1787784656.371704:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784656.371718:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668334592:0@lo:400:kworker.0 00000100:00100000:0.0:1787784656.372409:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:0.0:1787784656.372422:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668334080:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784656.372452:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784656.372518:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668334080:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (2143us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784656.372525:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:2.0:1787784656.372590:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668334080:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1787784656.818026:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784657.841840:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784657.972954:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967741 is committed 00000001:00080000:1.0:1787784657.972984:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967742 is committed 00010000:00010000:3.0:1787784658.866675:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784659.441570:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402184f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668334848:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784659.441670:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784659.441813:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334848 00000100:00100000:3.0:1787784659.441824:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334848 00000100:00100000:3.0:1787784659.441909:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:3.0:1787784659.441921:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668334848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784659.441988:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784659.442050:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668334848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 131us (380us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784659.442070:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:3.0:1787784659.442134:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402184f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668334848:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784659.442288:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784659.722305:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784659.723717:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784659.723760:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784659.723801:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784659.723824:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784659.723849:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784659.723882:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784659.723918:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657982464 00000100:00100000:0.0:1787784659.723931:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657982464 00000100:00100000:0.0:1787784659.724002:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657982080 00000100:00100000:0.0:1787784659.724006:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657982080 00000100:00100000:0.0:1787784659.724031:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:0.0:1787784659.724041:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2356:x1874627657982464:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784659.724144:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2356:x1874627657982464:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 102us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784659.724179:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:0.0:1787784659.724205:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 272 00000100:00100000:0.0:1787784659.724211:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:2356:x1874627657982080:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784659.724243:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:2356:x1874627657982080:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784659.724250:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 272 00000100:00100000:0.0:1787784659.724274:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657982208 00000100:00100000:0.0:1787784659.724278:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657982208 00000100:00100000:0.0:1787784659.724303:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657982720 00000100:00100000:0.0:1787784659.724306:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657982720 00000100:00100000:0.0:1787784659.724320:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657982336 00000100:00100000:0.0:1787784659.724322:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657982336 00000100:00100000:0.0:1787784659.724335:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657982592 00000100:00100000:0.0:1787784659.724337:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657982592 00000100:00100000:0.0:1787784659.724351:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 215 00000100:00100000:0.0:1787784659.724357:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657982208:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784659.724395:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657982208:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (688us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784659.724401:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 215 00000100:00100000:0.0:1787784659.724414:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 216 00000100:00100000:0.0:1787784659.724421:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657982720:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784659.724442:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657982720:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (642us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784659.724449:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 216 00000100:00100000:0.0:1787784659.724460:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 217 00000100:00100000:0.0:1787784659.724476:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2356:x1874627657982336:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784659.724501:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2356:x1874627657982336:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 35us (652us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784659.724507:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 217 00000100:00100000:0.0:1787784659.724519:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 218 00000100:00100000:0.0:1787784659.724525:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657982592:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784659.724549:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657982592:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (667us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784659.724555:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 218 00000100:00080000:0.0:1787784659.724574:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784659.724579:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784659.724582:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784659.725231:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657981952 00000100:00100000:3.0:1787784659.725241:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657981952 00000100:00100000:3.0:1787784659.725293:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:3.0:1787784659.725301:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657981952:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784659.725351:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627657981952:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 51us (3063us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784659.725357:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 347 00010000:00010000:1.0:1787784659.889555:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784660.913443:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787784661.443557:0:17095:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 18:50:59 (1787784659) 00000100:00100000:2.0:1787784661.492289:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539131680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668334976:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784661.492402:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784661.492602:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668334976 00000100:00100000:0.0:1787784661.492619:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668334976 00000100:00100000:0.0:1787784661.492707:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:0.0:1787784661.492719:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668334976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784661.492768:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784661.492836:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668334976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784661.492844:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00080000:2.0:1787784661.493129:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784661.493208:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539131680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668334976:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784661.495817:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668335232:0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.495873:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668335104:0@lo:400:kworker.0 00000100:00100000:2.0:1787784661.495921:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784661.496395:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784661.496508:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668335232 00000100:00100000:3.0:1787784661.496518:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668335104 00000100:00100000:2.0:1787784661.496519:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668335232 00000100:00100000:3.0:1787784661.496530:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668335104 00000100:00100000:3.0:1787784661.496592:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:2.0:1787784661.496605:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:3.0:1787784661.496607:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668335104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784661.496623:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668335232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.496677:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668335104:12345-0@lo:400:kworker.0 Request processed in 74us (282us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784661.496686:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:2.0:1787784661.496708:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668335232:12345-0@lo:400:kworker.0 Request processed in 88us (787us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784661.496718:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00100000:3.0:1787784661.496762:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668335104:0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.497217:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402184f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668335360:0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.497260:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784661.497286:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668335360 00000100:00100000:3.0:1787784661.497289:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668335360 00000100:00100000:3.0:1787784661.497316:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:3.0:1787784661.497320:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668335360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.497346:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668335360:12345-0@lo:400:kworker.0 Request processed in 26us (85us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784661.497350:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:3.0:1787784661.497374:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402184f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668335360:0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.497416:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402184f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668335488:0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.497443:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784661.497497:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668335488 00000100:00100000:3.0:1787784661.497502:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668335488 00000100:00100000:3.0:1787784661.497545:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1787784661.497553:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668335488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.497598:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668335488:12345-0@lo:400:kworker.0 Request processed in 47us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784661.497608:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1787784661.497645:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668335232:0@lo:400:kworker.0 00000100:00100000:3.0:1787784661.497667:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402184000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668335616:0@lo:400:kworker.0 00000100:00080000:2.0:1787784661.497690:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784661.497699:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784661.497700:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784661.497732:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668335616 00000100:00100000:2.0:1787784661.497740:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668335616 00000100:00080000:3.0:1787784661.497843:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784661.498101:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402184f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668335488:0@lo:400:kworker.0 00000100:00100000:2.0:1787784661.498600:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:2.0:1787784661.498615:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668335616:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784661.498711:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668335616:12345-0@lo:400:kworker.0 Request processed in 101us (1013us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784661.498720:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:3.0:1787784661.499922:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402184000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668335616:0@lo:400:kworker.0 00000100:00100000:0.0:1787784661.701420:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.701884:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657982976 00000100:00100000:3.0:1787784661.701898:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657982976 00000100:00100000:3.0:1787784661.702000:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 676 00000100:00100000:3.0:1787784661.702011:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+29:21818:x1874627657982976:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784661.702070:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784661.702097:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47dc lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b083b expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.702360:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.702380:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.702398:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787784661.702659:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784661.705648:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.705685:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.705700:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706219:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc47f1 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706232:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc47f1 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706241:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc47f1 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784661.706253:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:1.0:1787784661.706447:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706467:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706475:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706481:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784661.706486:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706495:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706508:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:1.0:1787784661.706511:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784661.706516:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706523:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc47e3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706535:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706541:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706548:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706564:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784661.706567:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706575:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706584:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff9895029d1900) 00010000:00010000:1.0:1787784661.706586:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784661.706589:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706595:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc47ea lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706937:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc47dc lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b083b expref: 32 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706955:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc47f1 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b083b expref: 31 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.706963:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784661.706966:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784661.706968:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad800, rc 0) 00000100:00100000:1.0:1787784661.707122:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21818:x1874627657982976:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5111us (5715us total) trans 4294967743 rc 0/0 00000100:00100000:1.0:1787784661.707132:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 676 00000100:00100000:3.0:1787784661.713850:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784661.714079:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657983104 00000100:00100000:0.0:1787784661.714092:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657983104 00000100:00100000:0.0:1787784661.714197:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 27 00000100:00100000:0.0:1787784661.714210:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2354:x1874627657983104:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784661.714227:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784661.714253:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc47f1 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b083b expref: 31 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.714271:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784661.714276:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784661.714344:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+30:2354:x1874627657983104:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 138us (510us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784661.714416:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 27 00000100:00100000:0.0:1787784661.718449:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.718644:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657983232 00000100:00100000:2.0:1787784661.718664:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657983232 00000100:00100000:2.0:1787784661.719103:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 677 00000100:00100000:2.0:1787784661.719120:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:21818:x1874627657983232:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1787784661.719344:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:21818:x1874627657983232:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 227us (903us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784661.719365:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 677 00000100:00100000:0.0:1787784661.736555:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.737895:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657983360 00000100:00100000:3.0:1787784661.737912:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657983360 00000100:00080000:2.0:1787784661.738023:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784661.738041:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 678 00000100:00100000:3.0:1787784661.738054:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:21819:x1874627657983360:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784661.738080:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784661.738105:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0842 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.738197:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.738214:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.738227:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742262:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742288:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742301:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742602:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742611:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742617:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742623:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784661.742627:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742634:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742646:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784661.742649:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.742653:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742659:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4806 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742670:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742675:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742681:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742692:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784661.742695:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742701:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.742710:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff989505542200) 00010000:00010000:3.0:1787784661.742712:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.742715:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.743984:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc480d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.744046:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ff lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0842 expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.744057:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784661.744060:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.744064:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc47ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0842 expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.744072:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2000, rc 0) 00000100:00100000:3.0:1787784661.744243:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:21819:x1874627657983360:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6190us (7699us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784661.744252:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 678 00000100:00100000:3.0:1787784661.754607:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784661.755143:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657983616 00000100:00100000:0.0:1787784661.755184:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657983616 00000100:00100000:0.0:1787784661.755245:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 679 00000100:00100000:0.0:1787784661.755263:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:21819:x1874627657983616:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784661.755318:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784661.755345:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc46f5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b07fc expref: 71 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.755370:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:0.0:1787784661.755375:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784661.755661:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.755686:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 29 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.755694:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.756112:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 29 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.756130:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 29 pid: 5811 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784661.756144:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539130780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668335744:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784661.756296:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef900 going to sleep for 11 seconds 00000100:00100000:0.0:1787784661.763708:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.763743:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539130780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668335744:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784661.763774:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784661.763889:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657983744 00000100:00100000:1.0:1787784661.763910:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657983744 00010000:00010000:1.0:1787784661.763944:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 30 pid: 5811 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784661.763985:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:1.0:1787784661.763995:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:2357:x1874627657983744:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784661.764010:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784661.764017:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 30 pid: 5811 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.764030:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 30 pid: 5811 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.764063:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc471f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b080a expref: 30 pid: 5811 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.764078:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:1.0:1787784661.764084:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.764093:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.764101:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.764133:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784661.764225:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+29:2357:x1874627657983744:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 230us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784661.764243:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 190 00010000:00010000:3.0:1787784661.764439:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.764448:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.764482:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.764491:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.764498:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784661.766644:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 512 00010000:00010000:3.0:1787784661.766662:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.766672:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.766679:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.766685:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbde00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784661.766689:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.766696:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.766710:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784661.767192:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.767199:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.767205:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4814 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784661.767225:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dbdd8c66 reply state 0000000018c4e91e transno 4294967744 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784661.767233:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784661.772993:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+72:21819:x1874627657983616:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 17721us (18382us total) trans 4294967744 rc 0/0 00000100:00100000:0.0:1787784661.773006:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 679 00010000:00010000:1.0:1787784661.776603:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.776621:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.776630:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.776637:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784661.776642:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.776651:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.776668:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff989505542200) 00010000:00010000:1.0:1787784661.776671:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784661.776676:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784661.776683:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc481b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784661.795132:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.796013:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657983872 00000100:00100000:3.0:1787784661.796027:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657983872 00000100:00100000:2.0:1787784661.796254:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 680 00000100:00100000:2.0:1787784661.796265:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+71:21820:x1874627657983872:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784661.796301:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.796324:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4822 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0849 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.796390:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4829 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.796406:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4829 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.796420:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4829 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784661.796489:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784661.796494:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 19 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784661.796518:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4829 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.796542:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4822 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0849 expref: 73 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.796578:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4829 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0849 expref: 72 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.796586:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1300) 00010000:00010000:2.0:1787784661.796589:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784661.796592:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea000, rc 0) 00000100:00100000:2.0:1787784661.796678:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+72:21820:x1874627657983872:12345-192.168.203.43@tcp:101:bash.0 Request processed in 415us (1556us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784661.796686:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 680 00000100:00100000:0.0:1787784661.809251:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.810109:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984000 00000100:00100000:2.0:1787784661.810121:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984000 00000100:00100000:2.0:1787784661.810219:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 681 00000100:00100000:2.0:1787784661.810230:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+72:21820:x1874627657984000:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784661.810249:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.810268:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4830 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0850 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810346:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810361:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810371:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810386:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810405:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810412:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810570:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc4845 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810580:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc4845 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810587:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc4845 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784661.810790:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 512 00000004:00200000:2.0:1787784661.810795:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784661.810815:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc4845 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810827:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810834:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810840:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810846:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.810850:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810862:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810874:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1300) 00010000:00010000:2.0:1787784661.810877:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.810882:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810888:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4837 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810901:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810912:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810918:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810924:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.810927:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810933:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810940:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff989530cb1800) 00010000:00010000:2.0:1787784661.810942:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.810946:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.810962:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc483e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.811118:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4830 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0850 expref: 74 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.811136:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eae00/0xb0ce63137acc4845 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0850 expref: 73 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.811143:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.811204:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784661.811207:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eae00, rc 0) 00000100:00100000:2.0:1787784661.811266:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21820:x1874627657984000:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1037us (2032us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784661.811284:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 681 00000100:00100000:1.0:1787784661.818736:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.818910:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984128 00000100:00100000:2.0:1787784661.818919:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984128 00000100:00100000:2.0:1787784661.818973:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 682 00000100:00100000:2.0:1787784661.818983:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21820:x1874627657984128:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784661.819003:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.819021:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc484c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0857 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819084:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819098:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819123:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819332:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819342:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819350:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819420:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819428:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819434:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819440:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.819444:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819463:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819475:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.819478:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.819482:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819494:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4853 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819507:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819513:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819519:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819523:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.819526:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819532:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819544:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989530cb1800) 00010000:00010000:2.0:1787784661.819547:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.819550:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819555:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc485a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819585:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc484c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0857 expref: 74 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819605:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.819608:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.819612:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc484c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0857 expref: 74 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.819621:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb400, rc 0) 00000100:00100000:2.0:1787784661.820608:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21820:x1874627657984128:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1619us (1872us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784661.820619:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 682 00000100:00100000:1.0:1787784661.843403:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.843524:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984256 00000100:00100000:2.0:1787784661.843533:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984256 00000100:00100000:2.0:1787784661.843596:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 683 00000100:00100000:2.0:1787784661.843607:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+29:21821:x1874627657984256:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784661.843627:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.843647:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4861 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b085e expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.843705:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.843718:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.843729:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784661.843784:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784661.843801:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 19 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784661.843829:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.843852:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4861 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 31 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.843870:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 30 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.843878:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1300) 00010000:00010000:2.0:1787784661.843881:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784661.843884:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea400, rc 0) 00000100:00100000:2.0:1787784661.843943:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+30:21821:x1874627657984256:12345-192.168.203.43@tcp:101:rm.0 Request processed in 338us (549us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784661.843950:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 683 00000100:00100000:0.0:1787784661.847185:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.847276:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984384 00000100:00100000:3.0:1787784661.847286:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984384 00000100:00100000:3.0:1787784661.847357:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 684 00000100:00100000:3.0:1787784661.847468:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+30:21821:x1874627657984384:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784661.847489:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784661.847510:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc486f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0865 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847570:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847585:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847596:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847611:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847636:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847643:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847749:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4884 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847758:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4884 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847765:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4884 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784661.847796:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 512 00000004:00200000:3.0:1787784661.847800:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784661.847821:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4884 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847832:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847839:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847845:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847850:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784661.847854:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847867:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847879:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1300) 00010000:00010000:3.0:1787784661.847883:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.847887:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847895:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4876 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847909:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847916:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847922:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847927:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784661.847930:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847936:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847943:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff989505542800) 00010000:00010000:3.0:1787784661.847946:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.847948:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847963:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc487d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.847993:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc486f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0865 expref: 32 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.848007:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2000/0xb0ce63137acc4884 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0865 expref: 31 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.848015:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:3.0:1787784661.848017:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784661.848020:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2000, rc 0) 00000100:00100000:3.0:1787784661.848081:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21821:x1874627657984384:12345-192.168.203.43@tcp:101:rm.0 Request processed in 714us (940us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784661.848088:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 684 00000100:00100000:1.0:1787784661.857920:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.858017:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984512 00000100:00100000:2.0:1787784661.858026:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984512 00000100:00100000:2.0:1787784661.858089:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 685 00000100:00100000:2.0:1787784661.858099:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21821:x1874627657984512:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784661.858118:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.858139:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc488b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b086c expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858384:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858407:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858429:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858445:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858461:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858468:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858531:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858538:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858545:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858551:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.858555:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858563:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858575:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.858578:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.858583:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858595:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4892 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858608:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858614:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858619:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858624:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.858627:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858633:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858640:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989530cb1800) 00010000:00010000:2.0:1787784661.858642:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.858646:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858652:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc4899 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858684:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc488b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b086c expref: 32 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858706:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.858709:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.858713:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc488b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b086c expref: 32 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.858723:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb400, rc 0) 00000100:00100000:2.0:1787784661.858845:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21821:x1874627657984512:12345-192.168.203.43@tcp:101:rm.0 Request processed in 747us (932us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784661.858853:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 685 00000100:00100000:1.0:1787784661.862034:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.862346:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984640 00000100:00100000:2.0:1787784661.862356:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984640 00000100:00100000:2.0:1787784661.862467:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 686 00000100:00100000:2.0:1787784661.862478:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21821:x1874627657984640:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784661.862496:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.862516:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc48a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0873 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862582:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862597:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862609:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862623:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862640:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862647:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862721:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862729:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862735:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862742:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.862746:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862754:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862765:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.862768:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.862773:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862779:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862794:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862806:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862812:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862818:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.862821:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862828:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862837:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989530cb1e00) 00010000:00010000:2.0:1787784661.862839:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.862842:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862848:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862880:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc48a0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0873 expref: 32 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862890:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.862891:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.862895:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc48a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0873 expref: 32 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.862916:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eaa00, rc 0) 00000100:00100000:2.0:1787784661.863065:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21821:x1874627657984640:12345-192.168.203.43@tcp:101:rm.0 Request processed in 588us (1039us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784661.863074:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 686 00000100:00100000:0.0:1787784661.881435:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.882960:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984768 00000100:00100000:2.0:1787784661.882975:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984768 00000100:00100000:2.0:1787784661.883162:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 687 00000100:00100000:2.0:1787784661.883173:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21822:x1874627657984768:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784661.883193:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.883215:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc48b5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b087a expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883351:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883366:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883378:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883392:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883400:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883407:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883553:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883562:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883567:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883573:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.883577:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883585:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883599:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.883602:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.883606:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883611:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc48bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883624:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883629:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883636:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883647:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.883650:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883657:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883665:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989530cb1e00) 00010000:00010000:2.0:1787784661.883668:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.883725:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883733:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc48c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883767:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc48b5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b087a expref: 32 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883777:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.883780:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.883783:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc48b5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b087a expref: 32 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.883792:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eaa00, rc 0) 00000100:00100000:2.0:1787784661.884078:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21822:x1874627657984768:12345-192.168.203.43@tcp:101:cp.0 Request processed in 905us (2651us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784661.884087:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 687 00000100:00100000:1.0:1787784661.887564:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.887651:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657984896 00000100:00100000:3.0:1787784661.887661:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657984896 00000100:00100000:3.0:1787784661.888002:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 688 00000100:00100000:3.0:1787784661.888012:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21822:x1874627657984896:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784661.888032:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784661.888107:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc48ca lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0881 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888185:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888199:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888221:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888235:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888242:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888248:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888369:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888376:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888382:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888388:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784661.888392:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888399:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888411:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:3.0:1787784661.888414:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.888418:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888430:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc48d1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888443:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888506:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888514:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888519:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784661.888523:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888529:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888538:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989505542800) 00010000:00010000:3.0:1787784661.888541:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.888544:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888550:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc48d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888584:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc48ca lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0881 expref: 32 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888606:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:3.0:1787784661.888608:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784661.888613:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc48ca lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0881 expref: 32 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.888623:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3a00, rc 0) 00000100:00100000:3.0:1787784661.888794:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21822:x1874627657984896:12345-192.168.203.43@tcp:101:cp.0 Request processed in 783us (1237us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784661.888803:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 688 00000100:00100000:1.0:1787784661.893425:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784661.893572:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657985152 00000100:00100000:0.0:1787784661.893581:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657985152 00000100:00100000:0.0:1787784661.893662:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 689 00000100:00100000:0.0:1787784661.893824:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21822:x1874627657985152:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784661.893846:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784661.893867:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc48df lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0888 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.893991:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.894006:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.894017:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.894031:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.894229:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.894236:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784661.895570:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xe:0x0], index 0: 0 00010000:00010000:0.0:1787784661.895855:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895876:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895886:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895922:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895930:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895936:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895943:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784661.895947:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895954:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895965:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:0.0:1787784661.895968:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784661.895973:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.895979:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc48e6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784661.895990:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000905afdb4 reply state 0000000078a4dff8 transno 4294967745 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784661.896004:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784661.896355:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc48df lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 34 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.896374:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.896383:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:0.0:1787784661.896385:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784661.896388:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbde00, rc 0) 00000100:00100000:0.0:1787784661.896580:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21822:x1874627657985152:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2908us (3162us total) trans 4294967745 rc 0/0 00000100:00100000:0.0:1787784661.896590:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 689 00000100:00100000:0.0:1787784661.901132:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.901422:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657985280 00000100:00100000:3.0:1787784661.901432:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657985280 00000100:00100000:3.0:1787784661.901515:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 172 00000100:00100000:3.0:1787784661.901525:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627657985280:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784661.901543:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784661.901560:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b088f expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784661.901588:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1787784661.901607:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=452s v=(0 0) 00010000:00010000:3.0:1787784661.901611:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784661.901621:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.901639:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.901680:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3a00, rc 0) 00000100:00100000:3.0:1787784661.901739:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657985280:12345-192.168.203.43@tcp:101:cp.0 Request processed in 215us (615us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784661.901748:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 172 00000100:00080000:0.0:1787784661.902083:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784661.905805:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.905984:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657985408 00000100:00100000:3.0:1787784661.905993:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657985408 00000100:00100000:3.0:1787784661.906060:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:3.0:1787784661.906071:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:21822:x1874627657985408:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784661.906090:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784661.906107:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0896 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.906139:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.906178:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784661.906182:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784661.906188:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 4/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.906223:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.906237:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 10 pid: 10688 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784661.906429:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668335872:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784661.906691:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9ccc0 going to sleep for 11 seconds 00010000:00010000:2.0:1787784661.907573:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.907595:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.907603:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.907609:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.907614:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.907621:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.907637:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989503a6d200) 00010000:00010000:2.0:1787784661.907641:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.907645:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.907651:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc48ed lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784661.910307:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.910474:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668335872:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784661.910507:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.910525:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2a00, rc 0) 00000100:00100000:3.0:1787784661.910580:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:21822:x1874627657985408:12345-192.168.203.43@tcp:101:cp.0 Request processed in 4511us (4784us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784661.910589:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:2.0:1787784661.910730:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657985536 00000100:00100000:2.0:1787784661.910737:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657985536 00010000:00010000:2.0:1787784661.910765:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 10 pid: 10688 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784661.910788:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 191 00000100:00100000:2.0:1787784661.910795:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657985536:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784661.910807:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784661.910840:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 10 pid: 10688 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.910853:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 10 pid: 10688 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.910873:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000498deaed/0xb0ce63137acc4902 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b088f expref: 10 pid: 10688 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.910886:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784661.910890:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784661.910897:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.910907:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.910937:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784661.910955:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668336000:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784661.911001:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895012549c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784661.915209:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784661.915299:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657985792 00000100:00100000:1.0:1787784661.915308:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657985792 00000100:00100000:1.0:1787784661.915360:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 141 00000100:00100000:1.0:1787784661.915370:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21822:x1874627657985792:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784661.916085:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21822:x1874627657985792:12345-192.168.203.43@tcp:35:cp.0 Request processed in 714us (882us total) trans 4294967746 rc 0/0 00000100:00100000:1.0:1787784661.916094:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 141 00000100:00100000:2.0:1787784661.916997:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668336000:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784661.917020:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784661.917064:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627657985536:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 6268us (6765us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784661.917092:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 191 00010000:00010000:1.0:1787784661.942426:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784661.957600:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784661.957699:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657985920 00000100:00100000:2.0:1787784661.957709:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657985920 00000100:00100000:2.0:1787784661.957765:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 690 00000100:00100000:2.0:1787784661.957775:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21825:x1874627657985920:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784661.957794:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784661.957813:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4910 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b089d expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.957873:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.957886:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.957896:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.957909:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.957916:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.957922:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958014:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4925 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958040:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4925 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958047:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4925 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784661.958081:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 158 00000004:00200000:2.0:1787784661.958085:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784661.958101:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4925 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958111:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4925 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958119:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958125:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958131:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958138:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.958142:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958291:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958303:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:2.0:1787784661.958312:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.958317:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958323:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4917 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958337:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958342:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958348:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958354:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784661.958357:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958363:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958370:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989530cb1e00) 00010000:00010000:2.0:1787784661.958373:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784661.958375:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958380:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc491e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958409:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4910 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b089d expref: 75 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958431:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4925 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b089d expref: 74 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784661.958438:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:2.0:1787784661.958440:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784661.958443:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb400, rc 0) 00000100:00100000:0.0:1787784661.958901:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:21825:x1874627657985920:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1126us (1309us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784661.958911:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 690 00000100:00100000:1.0:1787784661.963652:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784661.963719:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657986048 00000100:00100000:3.0:1787784661.963728:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657986048 00000100:00100000:3.0:1787784661.963789:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 174 00000100:00100000:3.0:1787784661.963798:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2355:x1874627657986048:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784661.963815:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784661.963831:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc492c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08a4 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784661.963883:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668336128:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784661.963988:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9ccc0 going to sleep for 11 seconds 00000100:00080000:0.0:1787784661.965509:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1787784661.973957:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1787784661.973965:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb mtime: 0 -> 1787784661 00002000:00010000:3.0:1787784661.973968:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb atime: 0 -> 1787784661 00002000:00010000:3.0:1787784661.973970:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb ctime: 0 -> 1787784661 00000100:00100000:3.0:1787784661.974009:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668336128:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784661.974045:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc492c lrc: 1/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08a4 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.974068:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc492c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08a4 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784661.974082:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784661.974222:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2355:x1874627657986048:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 10424us (10578us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784661.974232:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 174 00000100:00100000:2.0:1787784661.986258:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784661.997176:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657986176 00000100:00100000:0.0:1787784661.997192:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657986176 00000100:00100000:0.0:1787784661.997272:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 691 00000100:00100000:0.0:1787784661.997282:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:21825:x1874627657986176:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784661.997303:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784661.997342:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4933 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08ab expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.997424:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc493a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.997440:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc493a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.997462:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc493a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784661.997525:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 512 00000004:00200000:0.0:1787784661.997530:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784661.997555:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc493a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.997584:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4933 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ab expref: 76 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.997605:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc493a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ab expref: 75 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784661.997613:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:0.0:1787784661.997616:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784661.997619:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbcc00, rc 0) 00000100:00100000:0.0:1787784661.997679:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:21825:x1874627657986176:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 398us (11430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784661.997686:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 691 00000100:00100000:1.0:1787784662.001593:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784662.001646:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657986304 00000100:00100000:3.0:1787784662.001657:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657986304 00000100:00100000:3.0:1787784662.001712:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 175 00000100:00100000:3.0:1787784662.001724:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657986304:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784662.001742:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784662.001761:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4941 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08b2 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784662.001832:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668336256:192.168.203.43@tcp:106: 00000040:00080000:0.0:1787784662.001925:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000100:00100000:3.0:1787784662.001926:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9ccc0 going to sleep for 11 seconds 00000040:00080000:0.0:1787784662.001943:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784662.002138:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 1 00000040:00080000:0.0:1787784662.002166:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784662.005752:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668336256:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784662.005800:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4941 lrc: 1/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08b2 expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784662.005860:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4941 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08b2 expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784662.005875:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784662.005941:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657986304:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4218us (4357us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784662.005951:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 175 00000100:00100000:2.0:1787784662.011544:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784662.011673:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657986560 00000100:00100000:3.0:1787784662.011683:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657986560 00000100:00100000:3.0:1787784662.011742:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 692 00000100:00100000:3.0:1787784662.011753:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:21825:x1874627657986560:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784662.012016:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].16, 136 off10368 00000100:00100000:2.0:1787784662.012542:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:21825:x1874627657986560:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 788us (1007us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784662.012552:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 692 00000040:00080000:2.0:1787784662.679207:0:17144:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784662.679216:0:17144:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784662.679261:0:17144:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 17 count 2 00000040:00080000:2.0:1787784662.679268:0:17144:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784662.962492:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784663.025383:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784663.025392:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784663.025437:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 17 count 2 00000040:00080000:0.0:1787784663.025455:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784663.025805:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128b40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668336384:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784663.025909:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518ed6e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784663.030333:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128b40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668336384:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784663.032873:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.033108:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657986688 00000100:00100000:3.0:1787784663.033118:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657986688 00000100:00100000:3.0:1787784663.033197:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 693 00000100:00100000:3.0:1787784663.033206:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21836:x1874627657986688:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000100:00080000:1.0:1787784663.033342:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00000080:3.0:1787784663.033431:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x12:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784663.033553:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21836:x1874627657986688:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 347us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784663.033562:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 693 00000100:00100000:1.0:1787784663.051485:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.051699:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657986944 00000100:00100000:3.0:1787784663.051711:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657986944 00000100:00100000:3.0:1787784663.051764:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 694 00000100:00100000:3.0:1787784663.051787:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21836:x1874627657986944:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784663.051807:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784663.051828:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4948 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08b9 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.051932:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc494f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.051946:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc494f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.051958:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc494f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784663.051968:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 158 00010000:00010000:3.0:1787784663.052132:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4948 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08b9 expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.052176:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc494f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08b9 expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.052185:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:3.0:1787784663.052188:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784663.052192:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3400, rc 0) 00000100:00100000:3.0:1787784663.052242:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:21836:x1874627657986944:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 456us (766us total) trans 4294967747 rc 0/0 00000100:00100000:3.0:1787784663.052260:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 694 00000100:00100000:1.0:1787784663.064125:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784663.064194:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784663.064230:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657987200 00000100:00100000:1.0:1787784663.064238:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657987200 00000100:00100000:1.0:1787784663.064307:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 28 00000100:00100000:1.0:1787784663.064317:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+36:2354:x1874627657987200:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784663.064334:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784663.064361:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc494f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08b9 expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.064402:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:1.0:1787784663.064407:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784663.064694:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:2354:x1874627657987200:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 378us (498us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784663.064703:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 28 00000100:00100000:3.0:1787784663.066175:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657987072 00000100:00100000:3.0:1787784663.066185:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657987072 00000100:00100000:3.0:1787784663.066262:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 176 00000100:00100000:3.0:1787784663.066273:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657987072:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784663.066314:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.066333:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.066342:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xe:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1787784663.066347:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784663.066352:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 4/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.066387:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.066401:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 9 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784663.066416:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668336512:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784663.066517:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d900 going to sleep for 11 seconds 00000100:00080000:1.0:1787784663.067136:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784663.072263:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.072300:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657987328 00000100:00100000:3.0:1787784663.072309:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657987328 00002000:00100000:3.0:1787784663.072359:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xe:0x0] (0->157) req@ffff989402186580 x1874627657987328/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:544/0 lens 648/0 e 0 to 0 dl 1787784674 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784663.072388:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402186580 with x1874627657987328 ext(0->157) 00010000:00010000:3.0:1787784663.072398:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 10 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.072415:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 10 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784663.072423:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989402186580 00000100:00100000:3.0:1787784663.072461:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 6 00000100:00100000:3.0:1787784663.072471:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402186580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657987328:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784663.073351:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668336512:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784663.073379:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784663.075138:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402186580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627657987328:12345-192.168.203.43@tcp:4:cp.0 Request processed in 474us (691us total) trans 4294967325 rc 0/0 00000100:00100000:3.0:1787784663.075167:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 6 00002000:00100000:3.0:1787784663.075180:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xe:0x0] (0->157) req@ffff989402186580 x1874627657987328/t4294967325(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:544/0 lens 648/448 e 0 to 0 dl 1787784674 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784663.075190:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402186580 with x1874627657987328 ext(0->157) 00010000:00010000:3.0:1787784663.075198:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 12 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.075221:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 12 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784663.075229:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989402186580 00000100:00100000:3.0:1787784663.078283:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784663.078393:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657987456 00000100:00100000:2.0:1787784663.078402:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657987456 00010000:00010000:2.0:1787784663.078437:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 12 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784663.078496:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 192 00000100:00100000:2.0:1787784663.078512:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627657987456:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784663.078527:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784663.078549:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 12 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.078562:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 12 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.078586:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d70fd6c9/0xb0ce63137acc4909 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0896 expref: 12 pid: 6671 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.078601:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784663.078605:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784663.078610:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.078619:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.078652:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784663.078696:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657987456:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 186us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784663.078704:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 192 00010000:00010000:1.0:1787784663.079143:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079182:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079220:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079229:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079237:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079244:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784663.079249:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079272:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079285:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784663.079292:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa2943a3/0xb0ce63137acc495d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784663.079348:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627657987072:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 13075us (15231us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784663.079356:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 176 00000100:00100000:0.0:1787784663.086567:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784663.086661:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657987712 00000100:00100000:2.0:1787784663.086730:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657987712 00000100:00100000:2.0:1787784663.086780:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 695 00000100:00100000:2.0:1787784663.086791:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21836:x1874627657987712:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784663.086913:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21836:x1874627657987712:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 124us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784663.086920:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 695 00000100:00100000:1.0:1787784663.094337:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784663.094465:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657987840 00000100:00100000:0.0:1787784663.094475:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657987840 00000100:00100000:0.0:1787784663.094529:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:0.0:1787784663.094540:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627657987840:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784663.094557:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784663.094575:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08c0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784663.094601:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 158, mtime: 0x6a8f6dd5, atime: 0x0, ctime: 0x6a8f6dd5, blocks: 0x1 00010000:00010000:0.0:1787784663.094618:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=454s v=(0 0) 00010000:00010000:0.0:1787784663.094622:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784663.094632:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784663.094649:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784663.094681:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc400, rc 0) 00000100:00100000:0.0:1787784663.094732:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627657987840:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 193us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784663.094740:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:0.0:1787784663.105395:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.105563:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657988096 00000100:00100000:3.0:1787784663.105573:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657988096 00000100:00100000:3.0:1787784663.105627:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 696 00000100:00100000:3.0:1787784663.105637:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21836:x1874627657988096:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784663.105741:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21836:x1874627657988096:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 106us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784663.105750:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 696 00000001:00080000:3.0:1787784663.110425:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967745 is committed 00000001:00080000:3.0:1787784663.110506:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967746 is committed 00000001:00080000:3.0:1787784663.110517:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967744 is committed 00000100:00100000:0.0:1787784663.112842:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.112928:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657988224 00000100:00100000:3.0:1787784663.112939:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657988224 00002000:00100000:3.0:1787784663.112998:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xe:0x0] (0->4095) req@ffff989539128f00 x1874627657988224/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:544/0 lens 488/0 e 0 to 0 dl 1787784674 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784663.113010:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539128f00 with x1874627657988224 ext(0->4095) 00002000:00010000:3.0:1787784663.113016:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989539128f00 00000100:00100000:3.0:1787784663.113050:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 56 00000100:00100000:3.0:1787784663.113061:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657988224:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784663.113276:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657988224:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 216us (443us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784663.113284:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 56 00002000:00100000:3.0:1787784663.113291:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xe:0x0] (0->4095) req@ffff989539128f00 x1874627657988224/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:544/0 lens 488/4536 e 0 to 0 dl 1787784674 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784663.113301:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539128f00 with x1874627657988224 ext(0->4095) 00002000:00010000:3.0:1787784663.113304:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989539128f00 00000100:00100000:1.0:1787784663.127705:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784663.127794:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657988352 00000100:00100000:2.0:1787784663.127804:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657988352 00000100:00100000:2.0:1787784663.127902:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 697 00000100:00100000:2.0:1787784663.127913:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+32:21836:x1874627657988352:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784663.127932:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784663.127962:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc496b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08c7 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.127997:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.128018:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.128031:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.128131:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc496b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 34 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.128195:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 33 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.128374:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:2.0:1787784663.128377:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784663.128387:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea600, rc 0) 00000100:00100000:2.0:1787784663.128516:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21836:x1874627657988352:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 605us (818us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784663.128525:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 697 00000100:00100000:0.0:1787784663.136266:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.136327:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657988480 00000100:00100000:3.0:1787784663.136337:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657988480 00000100:00100000:3.0:1787784663.136392:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 698 00000100:00100000:3.0:1787784663.136401:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21836:x1874627657988480:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784663.136433:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x12:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784663.136616:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21836:x1874627657988480:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 215us (357us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784663.136624:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 698 00000100:00100000:1.0:1787784663.140830:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.140916:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657988608 00000100:00100000:3.0:1787784663.140924:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657988608 00000100:00100000:3.0:1787784663.140986:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:3.0:1787784663.140996:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657988608:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784663.141047:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141065:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784663.141070:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784663.141076:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141088:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141115:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141123:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141184:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141193:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784663.141198:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141207:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141223:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.141231:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce089f13/0xb0ce63137acc4979 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784663.141365:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657988608:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 368us (545us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784663.141374:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:1.0:1787784663.143983:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784663.144060:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657988864 00000100:00100000:3.0:1787784663.144068:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657988864 00000100:00100000:3.0:1787784663.144121:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 699 00000100:00100000:3.0:1787784663.144130:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21836:x1874627657988864:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784663.144225:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.144241:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 33 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.144267:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.144309:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 33 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.144324:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 33 pid: 5812 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784663.144337:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402184780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668336640:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784663.144397:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784663.147277:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402184780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668336640:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784663.147314:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784663.149229:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784663.150265:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657988992 00000100:00100000:2.0:1787784663.150276:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657988992 00010000:00010000:2.0:1787784663.150318:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 34 pid: 5812 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784663.150358:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:2.0:1787784663.150368:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2356:x1874627657988992:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784663.150382:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784663.150389:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 34 pid: 5812 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.150402:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 34 pid: 5812 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.150424:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4972 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c7 expref: 34 pid: 5812 timeout: 554 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.150454:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:2.0:1787784663.150459:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.150487:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.150494:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784663.150524:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784663.150582:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+33:2356:x1874627657988992:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 215us (1361us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784663.150590:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 193 00010000:00010000:3.0:1787784663.150755:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.150765:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.150945:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.150953:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.150959:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.150981:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784663.150986:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.150993:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.151004:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:3.0:1787784663.151006:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784663.151011:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784663.151017:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4980 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784663.151178:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21836:x1874627657988864:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 7047us (7203us total) trans 4294967748 rc 0/0 00000100:00100000:3.0:1787784663.151186:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 699 00000040:00080000:1.0:1787784663.151440:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784663.151458:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784663.151948:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:1.0:1787784663.151956:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784663.154395:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784663.154503:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657989248 00000100:00100000:0.0:1787784663.154511:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657989248 00000100:00100000:0.0:1787784663.154559:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 142 00000100:00100000:0.0:1787784663.154568:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21836:x1874627657989248:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784663.154837:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21836:x1874627657989248:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 269us (448us total) trans 4294967749 rc 0/0 00000100:00100000:0.0:1787784663.154845:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 142 00010000:00010000:3.0:1787784663.985620:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784664.561877:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784664.561904:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402184780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668336768:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784664.561977:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784664.566653:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668336768 00000100:00100000:3.0:1787784664.566675:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668336768 00000100:00100000:3.0:1787784664.566846:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:3.0:1787784664.566858:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668336768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784664.566893:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 62980096 pending 1703936 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784664.567001:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668336768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 146us (5023us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784664.567010:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:3.0:1787784664.567086:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402184780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668336768:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784664.567225:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:1.0:1787784664.702096:0:17196:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784664.702105:0:17196:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784664.702139:0:17196:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 17 count 2 00000040:00080000:1.0:1787784664.702172:0:17196:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784664.797676:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784664.797764:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657989504 00000100:00100000:3.0:1787784664.797773:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657989504 00000100:00100000:3.0:1787784664.797849:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 700 00000100:00100000:3.0:1787784664.797860:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:21844:x1874627657989504:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:3.0:1787784664.797880:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784664.797901:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4987 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08ce expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.798014:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc498e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.798030:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc498e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.798041:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc498e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784664.798052:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 158 00010000:00010000:3.0:1787784664.798215:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4987 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ce expref: 77 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.798235:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc498e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ce expref: 76 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.798244:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:3.0:1787784664.798247:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784664.798250:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784664.798305:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:21844:x1874627657989504:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 448us (636us total) trans 4294967750 rc 0/0 00000100:00100000:3.0:1787784664.798322:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 700 00000100:00080000:0.0:1787784664.799240:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784664.806803:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784664.807080:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657989632 00000100:00100000:2.0:1787784664.807090:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657989632 00000100:00100000:2.0:1787784664.807181:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 29 00000100:00100000:2.0:1787784664.807192:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ee450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:2357:x1874627657989632:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784664.807208:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784664.807233:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc498e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ce expref: 76 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784664.807254:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:2.0:1787784664.807259:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784664.807310:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ee450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:2357:x1874627657989632:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 122us (517us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784664.807318:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 29 00000100:00100000:1.0:1787784664.811703:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784664.811787:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657989760 00000100:00100000:3.0:1787784664.811797:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657989760 00000100:00100000:3.0:1787784664.811868:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 179 00000100:00100000:3.0:1787784664.811877:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627657989760:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:3.0:1787784664.811894:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784664.811911:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08d5 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.811934:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784664.811938:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784664.811947:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.811964:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.811977:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2400, rc 0) 00000100:00100000:3.0:1787784664.812036:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627657989760:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 160us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784664.812043:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 179 00000100:00080000:0.0:1787784664.812220:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784664.823757:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784664.826954:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657989888 00000100:00100000:3.0:1787784664.826967:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657989888 00002000:00100000:3.0:1787784664.827034:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xe:0x0] (0->4095) req@ffff98951b534780 x1874627657989888/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:545/0 lens 488/0 e 0 to 0 dl 1787784675 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784664.827060:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b534780 with x1874627657989888 ext(0->4095) 00002000:00010000:3.0:1787784664.827067:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b534780 00000100:00100000:3.0:1787784664.827103:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 57 00000100:00100000:3.0:1787784664.827113:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2354:x1874627657989888:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:3.0:1787784664.827757:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2354:x1874627657989888:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 643us (4007us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784664.827767:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 57 00002000:00100000:3.0:1787784664.827776:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xe:0x0] (0->4095) req@ffff98951b534780 x1874627657989888/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:545/0 lens 488/4536 e 0 to 0 dl 1787784675 ref 1 fl Complete:/600/0 rc 4096/4096 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784664.827785:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b534780 with x1874627657989888 ext(0->4095) 00002000:00010000:3.0:1787784664.827789:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b534780 00000100:00100000:2.0:1787784664.831916:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784664.831997:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657990144 00000100:00100000:0.0:1787784664.832007:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657990144 00000100:00100000:0.0:1787784664.832076:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 143 00000100:00100000:0.0:1787784664.832094:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507678850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:21844:x1874627657990144:12345-192.168.203.43@tcp:35:md5sum.0 00000100:00100000:0.0:1787784664.832376:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507678850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:21844:x1874627657990144:12345-192.168.203.43@tcp:35:md5sum.0 Request processed in 283us (468us total) trans 4294967751 rc 0/0 00000100:00100000:0.0:1787784664.832384:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 143 00000100:00100000:2.0:1787784664.840625:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784664.840669:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784664.840697:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784664.840731:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784664.840753:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784664.840781:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657990528 00000100:00100000:2.0:1787784664.840790:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657990528 00000100:00100000:2.0:1787784664.840842:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 276 00000100:00100000:2.0:1787784664.840852:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2357:x1874627657990528:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784664.840926:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2357:x1874627657990528:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 76us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784664.840934:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 276 00000100:00100000:2.0:1787784664.840956:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657990784 00000100:00100000:2.0:1787784664.840960:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657990784 00000100:00100000:2.0:1787784664.840990:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657990400 00000100:00100000:2.0:1787784664.840993:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657990400 00000100:00100000:2.0:1787784664.841008:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657990656 00000100:00100000:2.0:1787784664.841011:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657990656 00000100:00100000:2.0:1787784664.841023:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 220 00000100:00100000:2.0:1787784664.841030:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657990784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784664.841057:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627657990784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784664.841064:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 220 00000100:00100000:2.0:1787784664.841076:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 221 00000100:00100000:2.0:1787784664.841082:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657990400:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784664.841105:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:2357:x1874627657990400:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (375us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784664.841111:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 221 00000100:00100000:2.0:1787784664.841122:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 222 00000100:00100000:2.0:1787784664.841135:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657990656:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784664.841200:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627657990656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 52us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784664.841206:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 222 00000100:00080000:2.0:1787784664.841230:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784664.841768:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657990272 00000100:00100000:3.0:1787784664.841775:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657990272 00000100:00100000:3.0:1787784664.841818:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:3.0:1787784664.841826:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627657990272:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784664.841866:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627657990272:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (1249us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784664.841873:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:2.0:1787784664.858814:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784664.858922:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657991040 00000100:00100000:1.0:1787784664.858931:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657991040 00000100:00100000:1.0:1787784664.858985:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 701 00000100:00100000:1.0:1787784664.858995:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:21845:x1874627657991040:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:1.0:1787784664.859055:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1787784664.859082:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb400/0xb0ce63137acc4925 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b089d expref: 76 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.859100:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:1.0:1787784664.859113:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc493a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ab expref: 75 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.859123:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:1.0:1787784664.859126:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784664.859178:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.859347:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.859359:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859420:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859448:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859456:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859766:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859777:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859786:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859815:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784664.859828:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 33 pid: 10850 timeout: 555 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784664.859841:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668336896:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784664.859935:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950631e600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784664.866037:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784664.866069:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657991168 00000100:00100000:3.0:1787784664.866077:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657991168 00010000:00010000:3.0:1787784664.866116:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 34 pid: 10850 timeout: 555 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784664.866198:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:3.0:1787784664.866209:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2357:x1874627657991168:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784664.866224:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784664.866232:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 34 pid: 10850 timeout: 555 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.866245:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 34 pid: 10850 timeout: 555 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.866266:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:3.0:1787784664.866271:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.866280:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.866289:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.866308:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784664.866355:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2357:x1874627657991168:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 148us (324us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784664.866363:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 194 00010000:00010000:3.0:1787784664.866473:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc48f4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0888 expref: 33 pid: 10850 timeout: 555 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784664.866487:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668336896:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784664.866515:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.866524:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.866530:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784664.867654:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].5, 64 off8512 00000004:00080000:3.0:1787784664.867666:0:5813:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 455954518385ns, next at 456954516117ns, rc = 0 00010000:00010000:3.0:1787784664.867727:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867740:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867764:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867772:0:5813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867777:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867783:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867805:0:5813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787784664.867823:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867839:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867846:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867851:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbdc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784664.867855:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867862:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867872:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:3.0:1787784664.867875:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784664.867878:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867884:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49b1 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867902:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867908:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867915:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867920:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784664.867928:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867935:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867943:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:3.0:1787784664.867945:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784664.867948:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.867953:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc49a3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784664.867963:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009406c131 reply state 000000000c54513a transno 4294967752 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784664.867972:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784664.868041:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:21845:x1874627657991040:12345-192.168.203.43@tcp:36:rm.0 Request processed in 9046us (9234us total) trans 4294967752 rc 0/0 00000100:00100000:3.0:1787784664.868049:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 701 00010000:00010000:3.0:1787784664.868081:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784664.868087:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.868092:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2e00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784664.868095:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.868101:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.868119:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:3.0:1787784664.868122:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784664.868126:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.868131:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc49b8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784664.868165:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784664.868167:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784664.868169:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784664.874600:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.874620:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.874641:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.874648:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784664.874653:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.874661:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.874678:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989503a6d500) 00010000:00010000:1.0:1787784664.874682:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784664.874686:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784664.874693:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49aa lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.010428:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784665.034295:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784665.034661:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657991296 00000100:00100000:0.0:1787784665.034672:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657991296 00000100:00100000:0.0:1787784665.034734:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 702 00000100:00100000:0.0:1787784665.034744:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:21862:x1874627657991296:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784665.034768:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784665.034787:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc49bf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08dc expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.034850:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49c6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.034864:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.034875:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784665.034928:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 7680 00000004:00200000:0.0:1787784665.034932:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784665.034955:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.034980:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbca00/0xb0ce63137acc49bf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08dc expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.034997:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08dc expref: 76 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.035005:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:0.0:1787784665.035031:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784665.035036:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784665.035092:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:21862:x1874627657991296:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 349us (806us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784665.035100:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 702 00000100:00100000:3.0:1787784665.107490:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784665.107712:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657991424 00000100:00100000:2.0:1787784665.107726:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657991424 00000100:00100000:2.0:1787784665.107798:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 703 00000100:00100000:2.0:1787784665.107809:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21868:x1874627657991424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784665.107827:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784665.107848:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc49cd lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08e3 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.107910:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.107924:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.107935:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.107948:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.107956:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.107963:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787784665.108354:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784665.108679:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc49e2 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108690:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc49e2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108698:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc49e2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784665.108772:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00000004:00200000:2.0:1787784665.108776:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784665.108839:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc49e2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108850:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108857:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108863:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108870:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784665.108874:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108881:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108892:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:2.0:1787784665.108901:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784665.108907:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108913:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc49d4 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108924:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108931:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108936:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108940:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784665.108942:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108947:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108954:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff989530cb1e00) 00010000:00010000:2.0:1787784665.108957:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784665.108960:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108964:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc49db lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.108994:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc49cd lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08e3 expref: 35 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.109021:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea200/0xb0ce63137acc49e2 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08e3 expref: 34 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.109028:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784665.109030:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784665.109033:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea200, rc 0) 00000100:00100000:2.0:1787784665.109096:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21868:x1874627657991424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1289us (1615us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784665.110712:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 703 00000100:00100000:3.0:1787784665.115574:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784665.115887:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657991552 00000100:00100000:0.0:1787784665.115896:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657991552 00000100:00100000:0.0:1787784665.115949:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 704 00000100:00100000:0.0:1787784665.115958:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21868:x1874627657991552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784665.115976:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784665.115997:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49e9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08ea expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.116181:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49e9 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ea expref: 35 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.116491:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff989503a6dd00) 00010000:00010000:0.0:1787784665.116495:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784665.116501:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc49e9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ea expref: 35 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.116518:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbdc00, rc 0) 00000100:00100000:0.0:1787784665.116570:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21868:x1874627657991552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 613us (1007us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784665.116578:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 704 00000100:00100000:3.0:1787784665.119612:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784665.119700:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657991808 00000100:00100000:0.0:1787784665.119720:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657991808 00000100:00100000:0.0:1787784665.119822:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 705 00000100:00100000:0.0:1787784665.119831:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:21868:x1874627657991808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784665.119851:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784665.119854:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787784665.119880:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc49c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08dc expref: 76 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.119898:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:0.0:1787784665.119903:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784665.119916:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc49f0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08f1 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120043:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120056:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120066:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120078:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120085:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120091:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120822:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120834:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120842:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784665.120850:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 0 00010000:00010000:0.0:1787784665.120873:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120880:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120886:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120893:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbcc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784665.120896:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120903:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120912:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:0.0:1787784665.120915:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784665.120918:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.120934:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc49f7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784665.120958:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000084b922fe reply state 000000007e85fb44 transno 4294967753 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784665.120968:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784665.120996:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc49f0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 78 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.121009:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 77 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.121017:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6db00) 00010000:00010000:0.0:1787784665.121020:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784665.121022:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd600, rc 0) 00000100:00100000:0.0:1787784665.121097:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:21868:x1874627657991808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1267us (1494us total) trans 4294967753 rc 0/0 00000100:00100000:0.0:1787784665.121105:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 705 00000100:00100000:3.0:1787784665.126349:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784665.126442:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.126455:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.126462:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.126469:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbda00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784665.126474:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.126499:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.126515:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989503a6d800) 00010000:00010000:1.0:1787784665.126518:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784665.126523:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.126529:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbda00/0xb0ce63137acc49fe lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784665.133689:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657992064 00000100:00100000:2.0:1787784665.133711:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657992064 00000100:00100000:2.0:1787784665.133806:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 706 00000100:00100000:2.0:1787784665.133816:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:21868:x1874627657992064:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784665.133844:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784665.133866:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4a13 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08f8 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.133965:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.133981:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 78 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.133989:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.134348:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 78 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.134378:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 78 pid: 5813 timeout: 556 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784665.134392:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668337024:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784665.134610:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cfe180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784665.144835:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784665.146374:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657992192 00000100:00100000:0.0:1787784665.146387:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657992192 00010000:00010000:0.0:1787784665.146446:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 79 pid: 5813 timeout: 556 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784665.146492:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:0.0:1787784665.146502:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:2357:x1874627657992192:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784665.146518:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784665.146538:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 79 pid: 5813 timeout: 556 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.146551:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 79 pid: 5813 timeout: 556 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.146574:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6db00) 00010000:00010000:0.0:1787784665.146579:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.146588:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.146595:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.146616:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784665.146676:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:2357:x1874627657992192:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 176us (1851us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784665.146683:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 195 00010000:00010000:2.0:1787784665.149846:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4a05 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f1 expref: 78 pid: 5813 timeout: 556 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784665.149872:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668337024:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784665.149898:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.154635:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.154652:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784665.154675:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 0 00010000:00010000:3.0:1787784665.154950:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.154959:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.154966:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.154985:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784665.154989:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.154997:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.155014:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6db00) 00010000:00010000:3.0:1787784665.155017:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784665.155022:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.155028:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4a1a lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.155094:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4a13 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f8 expref: 78 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.155842:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:3.0:1787784665.155846:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784665.155852:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4a13 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08f8 expref: 78 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.155864:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea600, rc 0) 00000100:00100000:3.0:1787784665.155973:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:21868:x1874627657992064:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 22157us (29632us total) trans 4294967754 rc 301/301 00000100:00100000:3.0:1787784665.155982:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 706 00000100:00100000:1.0:1787784665.161031:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784665.161127:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657992448 00000100:00100000:0.0:1787784665.161136:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657992448 00000100:00100000:0.0:1787784665.161226:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 144 00000100:00100000:0.0:1787784665.161236:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:21868:x1874627657992448:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784665.161587:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:21868:x1874627657992448:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 351us (563us total) trans 4294967755 rc 0/0 00000100:00100000:0.0:1787784665.161595:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 144 00000100:00100000:2.0:1787784665.165979:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784665.166069:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657992576 00000100:00100000:3.0:1787784665.166092:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657992576 00000100:00100000:3.0:1787784665.166203:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 707 00000100:00100000:3.0:1787784665.166213:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:21868:x1874627657992576:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784665.166235:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784665.166257:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4a28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b08ff expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.166330:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4a28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ff expref: 79 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.166343:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4a28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ff expref: 79 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.166359:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989505542400) 00010000:00010000:3.0:1787784665.166362:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784665.166365:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2e00, rc 0) 00000100:00100000:3.0:1787784665.166411:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:21868:x1874627657992576:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 200us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784665.166418:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 707 00000100:00100000:1.0:1787784665.173675:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784665.173965:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657992832 00000100:00100000:3.0:1787784665.173975:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657992832 00000100:00100000:3.0:1787784665.174036:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 708 00000100:00100000:3.0:1787784665.174046:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:21868:x1874627657992832:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1787784665.184790:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.184816:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.184829:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.185054:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.185062:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.185086:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.185094:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784665.185098:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.185106:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.185118:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989505542400) 00010000:00010000:2.0:1787784665.185121:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787784665.185125:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784665.185132:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4a2f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784665.185250:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:21868:x1874627657992832:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 11201us (11582us total) trans 4294967756 rc 0/0 00000100:00100000:2.0:1787784665.185261:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 708 00000100:00100000:1.0:1787784665.190921:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784665.191004:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657993088 00000100:00100000:3.0:1787784665.191025:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657993088 00000100:00100000:3.0:1787784665.191081:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 709 00000100:00100000:3.0:1787784665.191091:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:21868:x1874627657993088:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784665.191599:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.191618:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784665.191631:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784665.191703:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967757 00000004:00010000:3.0:1787784665.191744:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005c31bf39 reply state 00000000e80f6180 transno 4294967758 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784665.191753:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784665.192495:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21868:x1874627657993088:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 1400us (1579us total) trans 4294967758 rc 0/0 00000100:00100000:3.0:1787784665.192504:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 709 00000020:00080000:3.0:1787784665.192509:0:5812:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1787784665.211548:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784665.211805:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657993344 00000100:00100000:2.0:1787784665.211816:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657993344 00000100:00100000:2.0:1787784665.211874:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 145 00000100:00100000:2.0:1787784665.211884:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21868:x1874627657993344:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784665.212117:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:21868:x1874627657993344:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 233us (578us total) trans 4294967759 rc 0/0 00000100:00100000:2.0:1787784665.212126:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 145 00010000:00010000:1.0:1787784665.212948:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.212961:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.212968:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.212974:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784665.212979:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.212987:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.213025:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989505542400) 00010000:00010000:1.0:1787784665.213029:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787784665.213033:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.213040:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3600/0xb0ce63137acc4a36 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784665.261708:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784665.261809:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657993600 00000100:00100000:0.0:1787784665.261831:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657993600 00000100:00100000:0.0:1787784665.261891:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 710 00000100:00100000:0.0:1787784665.261901:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21870:x1874627657993600:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:0.0:1787784665.261925:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784665.261945:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a3d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0906 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262020:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262033:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262045:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262059:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262066:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262073:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262243:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4a52 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262253:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4a52 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262259:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4a52 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784665.262275:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 158 00010000:00010000:0.0:1787784665.262373:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262381:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262388:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262394:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784665.262398:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262406:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262418:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:0.0:1787784665.262421:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784665.262424:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262651:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a44 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262666:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262672:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262689:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262694:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbdc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784665.262698:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262705:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262713:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff989503a6dd00) 00010000:00010000:0.0:1787784665.262716:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784665.262719:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262725:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262765:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a3d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0906 expref: 86 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262778:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4a52 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0906 expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.262785:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989505542400) 00010000:00010000:0.0:1787784665.262788:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784665.262791:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd000, rc 0) 00000100:00100000:0.0:1787784665.262888:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:21870:x1874627657993600:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 988us (1188us total) trans 4294967760 rc 0/0 00000100:00100000:0.0:1787784665.262907:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 710 00000100:00100000:3.0:1787784665.277618:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784665.277704:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657993856 00000100:00100000:1.0:1787784665.277713:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657993856 00000100:00100000:1.0:1787784665.277769:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 30 00000100:00100000:1.0:1787784665.277778:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:21870:x1874627657993856:12345-192.168.203.43@tcp:103:md5sum.0 00010000:00010000:1.0:1787784665.277792:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1787784665.277814:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4a52 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0906 expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.277834:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4a28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08ff expref: 84 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.277847:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989505542400) 00010000:00010000:1.0:1787784665.277855:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784665.277907:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21870:x1874627657993856:12345-192.168.203.43@tcp:103:md5sum.0 Request processed in 130us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784665.277914:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 30 00000100:00100000:3.0:1787784665.283877:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784665.283970:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657993984 00000100:00100000:0.0:1787784665.283980:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657993984 00000100:00100000:0.0:1787784665.284037:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 711 00000100:00100000:0.0:1787784665.284047:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21870:x1874627657993984:12345-192.168.203.43@tcp:58:md5sum.0 00010000:00010000:0.0:1787784665.284169:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.284196:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784665.284209:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784665.284397:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].17, 136 off10504 00000100:00100000:0.0:1787784665.284492:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21870:x1874627657993984:12345-192.168.203.43@tcp:58:md5sum.0 Request processed in 445us (621us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784665.284500:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 711 00000100:00100000:2.0:1787784665.292990:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784665.294294:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657994112 00000100:00100000:1.0:1787784665.294306:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657994112 00000100:00100000:1.0:1787784665.294387:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 712 00000100:00100000:1.0:1787784665.294397:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:21870:x1874627657994112:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:1.0:1787784665.294419:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784665.294457:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b090d expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.294515:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.294527:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b090d expref: 84 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.294547:0:10850:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.294559:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b090d expref: 84 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.294573:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6dd00) 00010000:00010000:1.0:1787784665.294576:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784665.294581:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b090d expref: 84 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784665.294589:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010aca00, rc 0) 00000100:00100000:1.0:1787784665.294658:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:21870:x1874627657994112:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 262us (1677us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784665.294665:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 712 00010000:00010000:3.0:1787784666.033471:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784666.225391:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784666.225402:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784666.225482:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 18 count 3 00000040:00080000:0.0:1787784666.225490:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784666.225612:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668337152:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784666.225717:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784666.230549:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668337152:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784666.232064:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.232346:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657994240 00000100:00100000:0.0:1787784666.232355:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657994240 00000100:00100000:0.0:1787784666.232436:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 713 00000100:00100000:0.0:1787784666.232446:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657994240:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784666.232479:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784666.232590:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657994240:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 145us (532us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.232598:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 713 00000100:00100000:1.0:1787784666.237649:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.237758:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657994368 00000100:00100000:2.0:1787784666.237767:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657994368 00000100:00100000:2.0:1787784666.237838:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 714 00000100:00100000:2.0:1787784666.237849:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657994368:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1787784666.237927:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.237946:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.237958:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238029:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238038:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238046:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238070:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238078:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238086:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784666.238182:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 158 00000004:00200000:2.0:1787784666.238186:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784666.238200:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238211:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238218:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238225:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238231:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784666.238235:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238243:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238255:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784666.238257:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784666.238262:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238268:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eba00/0xb0ce63137acc4a6e lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238280:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238306:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238312:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238317:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eaa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784666.238320:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238326:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238334:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x7d5aa0c7 (ffff989530cb1e00) 00010000:00010000:2.0:1787784666.238336:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784666.238339:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238344:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eaa00/0xb0ce63137acc4a75 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238359:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238365:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238371:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238375:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784666.238382:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238389:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238397:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6dd00) 00010000:00010000:2.0:1787784666.238400:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784666.238402:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.238408:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4a7c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.238493:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657994368:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 645us (854us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.238502:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 714 00000100:00100000:1.0:1787784666.252601:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.253084:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657994624 00000100:00100000:0.0:1787784666.253094:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657994624 00000100:00100000:0.0:1787784666.253175:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 715 00000100:00100000:0.0:1787784666.253186:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657994624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784666.253207:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784666.253229:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a83 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0914 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253322:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253338:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253350:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784666.253362:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 158 00010000:00010000:0.0:1787784666.253465:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253474:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253495:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253501:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784666.253505:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253513:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253524:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6dd00) 00010000:00010000:0.0:1787784666.253527:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784666.253531:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253537:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4a8a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253575:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a83 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0914 expref: 36 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253585:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784666.253587:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784666.253591:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4a83 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0914 expref: 36 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.253600:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbdc00, rc 0) 00000100:00100000:0.0:1787784666.253759:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21871:x1874627657994624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 574us (1167us total) trans 4294967761 rc 301/301 00000100:00100000:0.0:1787784666.253778:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 715 00000100:00100000:1.0:1787784666.268682:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.270945:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657994752 00000100:00100000:2.0:1787784666.270960:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657994752 00000100:00100000:2.0:1787784666.271358:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 716 00000100:00100000:2.0:1787784666.271372:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21871:x1874627657994752:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784666.271409:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784666.271575:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21871:x1874627657994752:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 206us (2901us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.271582:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 716 00000100:00100000:1.0:1787784666.281278:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.281689:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657995008 00000100:00100000:3.0:1787784666.281700:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657995008 00000100:00100000:3.0:1787784666.281761:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 717 00000100:00100000:3.0:1787784666.281771:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21871:x1874627657995008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784666.281794:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784666.281826:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4a98 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b091b expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784666.282747:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x10:0x0], index 1: 0 00010000:00010000:3.0:1787784666.283298:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4a9f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.283314:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4a9f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.283324:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4a9f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.283382:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4a98 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b091b expref: 38 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.283399:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4a9f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b091b expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.283408:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989505542d00) 00010000:00010000:3.0:1787784666.283410:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784666.283413:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2a00, rc 0) 00000100:00100000:3.0:1787784666.283591:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:21871:x1874627657995008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1820us (2322us total) trans 4294967762 rc 0/0 00000100:00100000:3.0:1787784666.283600:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 717 00000100:00100000:1.0:1787784666.293667:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.294614:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657995264 00000100:00100000:0.0:1787784666.294628:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657995264 00000100:00100000:0.0:1787784666.294691:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 718 00000100:00100000:0.0:1787784666.294700:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:21871:x1874627657995264:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784666.294767:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784666.294794:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4a9f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b091b expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.294825:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989505542d00) 00010000:00010000:0.0:1787784666.294840:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784666.294863:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.294886:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.294895:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.294910:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.294917:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.294924:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295394:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295403:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295409:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295415:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbdc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784666.295418:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295434:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295442:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff989530cb1a00) 00010000:00010000:0.0:1787784666.295445:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784666.295449:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295454:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4aad lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295463:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295469:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295474:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295486:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784666.295490:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295496:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295504:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x38fe7e97 (ffff989503a6d300) 00010000:00010000:0.0:1787784666.295506:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784666.295508:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.295514:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ab4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0x38fe7e97 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784666.295606:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:21871:x1874627657995264:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 905us (1948us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784666.295614:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 718 00000100:00100000:1.0:1787784666.298234:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.298286:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657995392 00000100:00100000:0.0:1787784666.298295:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657995392 00000100:00100000:0.0:1787784666.298351:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 719 00000100:00100000:0.0:1787784666.298361:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+36:21871:x1874627657995392:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784666.298381:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784666.298402:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4abb lrc: 2/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0922 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.298452:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.298481:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.298492:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.298581:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4abb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.298599:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 37 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.298607:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784666.298609:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784666.298612:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd000, rc 0) 00000100:00100000:0.0:1787784666.298665:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:21871:x1874627657995392:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 306us (439us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.298673:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 719 00000100:00100000:1.0:1787784666.302309:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.302415:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657995648 00000100:00100000:2.0:1787784666.302437:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657995648 00000100:00100000:2.0:1787784666.302502:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 720 00000100:00100000:2.0:1787784666.302512:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:21871:x1874627657995648:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784666.302618:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.302635:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.302648:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784666.303736:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].8, 88 off8752 00000004:00080000:2.0:1787784666.303751:0:10850:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 457390603315ns, next at 458390599987ns, rc = 0 00000004:00010000:2.0:1787784666.303925:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000021cb8854 reply state 000000004e469bd8 transno 4294967763 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.303936:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784666.305014:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+39:21871:x1874627657995648:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 2489us (2700us total) trans 4294967763 rc 0/0 00000100:00100000:2.0:1787784666.305025:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 720 00010000:00010000:2.0:1787784666.306876:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.306891:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.306898:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.306905:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784666.306909:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.306937:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.306954:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784666.306957:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787784666.306962:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.306967:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4ac9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784666.310451:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.311453:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657995776 00000100:00100000:3.0:1787784666.311466:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657995776 00000100:00100000:3.0:1787784666.311525:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 721 00000100:00100000:3.0:1787784666.311535:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:21871:x1874627657995776:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784666.311558:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784666.311591:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0929 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.311652:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.311665:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.311684:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:3.0:1787784666.311687:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784666.311692:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3c00, rc 0) 00000100:00100000:3.0:1787784666.311753:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:21871:x1874627657995776:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 219us (1311us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784666.311760:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 721 00000100:00100000:3.0:1787784666.321198:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.321279:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657995904 00000100:00100000:2.0:1787784666.321288:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657995904 00000100:00100000:2.0:1787784666.321352:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 180 00000100:00100000:2.0:1787784666.321362:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627657995904:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00000100:00080000:0.0:1787784666.321393:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784666.321683:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627657995904:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 322us (493us total) trans 4294967331 rc 0/0 00000100:00100000:2.0:1787784666.321691:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 180 00000100:00100000:0.0:1787784666.333978:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.334106:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657996160 00000100:00100000:2.0:1787784666.334117:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657996160 00000100:00100000:2.0:1787784666.334207:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 146 00000100:00100000:2.0:1787784666.334218:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:21871:x1874627657996160:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784666.334559:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:21871:x1874627657996160:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 343us (598us total) trans 4294967764 rc 0/0 00000100:00100000:2.0:1787784666.334568:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 146 00000100:00100000:1.0:1787784666.337137:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.337256:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657996416 00000100:00100000:0.0:1787784666.337264:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657996416 00000100:00100000:0.0:1787784666.337307:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 722 00000100:00100000:0.0:1787784666.337316:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:21871:x1874627657996416:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784666.337418:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:21871:x1874627657996416:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 103us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.337438:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 722 00000100:00100000:3.0:1787784666.350368:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.350625:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657996544 00000100:00100000:0.0:1787784666.350635:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657996544 00000100:00100000:0.0:1787784666.350688:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 723 00000100:00100000:0.0:1787784666.350698:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:21871:x1874627657996544:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784666.350718:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784666.350737:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4ad7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0930 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.350807:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ade lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.350831:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ade lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.350843:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ade lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784666.350875:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784666.350891:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ade lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.350913:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc600/0xb0ce63137acc4ad7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0930 expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.350929:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ade lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0930 expref: 41 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.350938:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784666.350941:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1787784666.350944:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784666.351018:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:21871:x1874627657996544:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 322us (659us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.351026:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 723 00000100:00100000:1.0:1787784666.353583:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.353664:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657996672 00000100:00100000:2.0:1787784666.353686:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657996672 00000100:00100000:2.0:1787784666.353738:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:2.0:1787784666.353747:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657996672:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784666.353764:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784666.353784:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0937 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784666.353819:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x10:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a8f6dd9, blocks: 0x1 00010000:00010000:2.0:1787784666.353839:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=457s v=(0 0) 00010000:00010000:2.0:1787784666.353843:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784666.353852:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.353869:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.353881:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea800, rc 0) 00000100:00100000:2.0:1787784666.353935:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657996672:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 188us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.353942:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:1.0:1787784666.357407:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.357494:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657996928 00000100:00100000:0.0:1787784666.357504:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657996928 00000100:00100000:0.0:1787784666.357556:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 724 00000100:00100000:0.0:1787784666.357567:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:21871:x1874627657996928:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784666.357721:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:21871:x1874627657996928:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 156us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.357730:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 724 00000100:00100000:3.0:1787784666.364097:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.364475:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657997056 00000100:00100000:2.0:1787784666.364485:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657997056 00000100:00100000:2.0:1787784666.364545:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:2.0:1787784666.364555:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:21871:x1874627657997056:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784666.364572:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784666.364590:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 2/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b093e expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.364640:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.364649:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784666.364653:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784666.364658:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 4/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 12 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.364691:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.364705:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 12 pid: 10688 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.364718:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668337280:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784666.364881:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cfe180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784666.369566:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668337280:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784666.369603:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 3/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 12 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.369624:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eaa00, rc 0) 00000100:00100000:2.0:1787784666.369691:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:21871:x1874627657997056:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5136us (5602us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.369700:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:2.0:1787784666.371992:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.372885:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657997184 00000100:00100000:0.0:1787784666.372895:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657997184 00010000:00010000:0.0:1787784666.373097:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 12 pid: 10688 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784666.373141:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:0.0:1787784666.373168:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627657997184:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784666.373186:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784666.373209:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 12 pid: 10688 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.373224:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 12 pid: 10688 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.373249:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eac19671/0xb0ce63137acc4ae5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0937 expref: 12 pid: 10688 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.373287:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784666.373291:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784666.373300:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.373311:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.373348:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784666.373364:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668337408:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784666.373605:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784666.381539:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.381620:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657997312 00000100:00100000:2.0:1787784666.381630:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657997312 00002000:00100000:2.0:1787784666.381681:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x10:0x0] (0->157) req@ffff9895406670c0 x1874627657997312/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:547/0 lens 648/0 e 0 to 0 dl 1787784677 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784666.381692:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895406670c0 with x1874627657997312 ext(0->157) 00002000:00010000:2.0:1787784666.381698:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9895406670c0 00000100:00100000:2.0:1787784666.381775:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 58 00000100:00100000:2.0:1787784666.381785:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406670c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627657997312:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1787784666.382301:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406670c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627657997312:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 515us (769us total) trans 4294967332 rc 0/0 00000100:00100000:2.0:1787784666.382310:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 58 00002000:00100000:2.0:1787784666.382318:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x10:0x0] (0->157) req@ffff9895406670c0 x1874627657997312/t4294967332(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:547/0 lens 648/448 e 0 to 0 dl 1787784677 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784666.382328:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895406670c0 with x1874627657997312 ext(0->157) 00002000:00010000:2.0:1787784666.382331:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9895406670c0 00000100:00100000:0.0:1787784666.382615:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668337408:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784666.382639:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784666.382663:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627657997184:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 9499us (10680us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.382673:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:1.0:1787784666.386011:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.386356:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657997440 00000100:00100000:0.0:1787784666.386365:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657997440 00000100:00100000:0.0:1787784666.386522:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 725 00000100:00100000:0.0:1787784666.386533:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:21871:x1874627657997440:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1787784666.419969:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967761 is committed 00000001:00080000:2.0:1787784666.419985:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967760 is committed 00000001:00080000:2.0:1787784666.420001:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967764 is committed 00000004:00080000:2.0:1787784666.420074:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 9 committed 00000004:00080000:2.0:1787784666.420104:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 6 committed 00000100:00100000:2.0:1787784666.420243:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784666.420261:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539379680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668337536:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787784666.420320:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784666.420557:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657997440:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 34022us (34559us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784666.420567:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 725 00000100:00100000:0.0:1787784666.420714:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784666.420725:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668337664:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1787784666.420879:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784666.420955:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668337536 00000100:00100000:1.0:1787784666.420964:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668337536 00000100:00100000:1.0:1787784666.421077:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668337664 00000100:00100000:1.0:1787784666.421081:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668337664 00000100:00100000:1.0:1787784666.421104:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1787784666.421112:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668337536:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1787784666.421134:0:10688:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:14 count 1 00010000:00010000:1.0:1787784666.421178:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.421198:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.421219:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.421227:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=2s v=(0 0) 00010000:00010000:1.0:1787784666.421232:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784666.421236:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 5/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.421265:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.421278:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 10 pid: 6671 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784666.421290:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a921c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668337792:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784666.421344:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.421355:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 10 pid: 6670 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784666.421364:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668337920:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784666.421387:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784666.421412:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1787784666.423843:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668337664:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1787784666.424432:0:6670:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:1.0:1787784666.424437:0:6670:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb mtime from disk: 0 -> 1787784665 00002000:00010000:1.0:1787784666.424440:0:6670:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1787784666.424442:0:6670:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb ctime from disk: 1787784665 -> 1787784665 00002000:00010000:1.0:1787784666.424445:0:6670:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:1.0:1787784666.424534:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668337664:12345-0@lo:2:osp-syn-1-0.0 Request processed in 709us (3652us total) trans 4294967333 rc 0/0 00000100:00100000:1.0:1787784666.424544:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00080000:0.0:1787784666.424707:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784666.424712:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:2.0:1787784666.425198:0:3279:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff98951b536580/2, rc 0, transno 4294967333 00000100:00100000:2.0:1787784666.425207:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668337664:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1787784666.426075:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.426167:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a921c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668337792:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784666.426180:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668337920:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784666.426211:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784666.426274:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657997696 00000100:00100000:3.0:1787784666.426281:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657997696 00010000:00010000:3.0:1787784666.426323:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 11 pid: 6670 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784666.426354:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 197 00000100:00100000:3.0:1787784666.426362:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657997696:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784666.426375:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784666.426380:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 11 pid: 6670 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.426401:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 11 pid: 6670 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.426439:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a691f5e2/0xb0ce63137acc4964 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08c0 expref: 11 pid: 6670 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.426455:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784666.426505:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627657997696:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 144us (434us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784666.426513:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 197 00000100:00080000:0.0:1787784666.426606:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784666.429243:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.429331:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657997824 00000100:00100000:3.0:1787784666.429339:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657997824 00000100:00100000:3.0:1787784666.429391:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 185 00000100:00100000:3.0:1787784666.429401:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539378f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627657997824:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1787784666.435540:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.436577:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657997568 00000100:00100000:3.0:1787784666.436588:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657997568 00010000:00010000:3.0:1787784666.436656:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 11 pid: 6671 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784666.436694:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 198 00000100:00100000:3.0:1787784666.436704:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627657997568:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784666.436720:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784666.436725:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 11 pid: 6671 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.436833:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 11 pid: 6671 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.436858:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001d26f97/0xb0ce63137acc499c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b08d5 expref: 11 pid: 6671 timeout: 557 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.436876:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784666.436881:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784666.436887:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.436898:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.436927:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784666.436979:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627657997568:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 276us (1448us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784666.436988:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 198 00010000:00010000:3.0:1787784666.440671:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.440694:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.440708:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.440793:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.440818:0:10688:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00080000:1.0:1787784666.441556:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784666.452886:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784666.452902:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.452918:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784666.452923:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.452935:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.452956:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 1/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.452965:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c1338a4/0xb0ce63137acc4af3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.452990:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784666.452992:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784666.452994:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784666.455633:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668337536:12345-0@lo:6:osp-syn-0-0.0 Request processed in 34514us (35305us total) trans 4294967326 rc 0/0 00000100:00100000:3.0:1787784666.455645:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000004:00080000:3.0:1787784666.455707:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989539379680/2, rc 0, transno 4294967326 00000100:00100000:3.0:1787784666.455713:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539379680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668337536:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1787784666.477609:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539378f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627657997824:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 48199us (48364us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.477622:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 185 00000001:00080000:1.0:1787784666.478698:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967333 is committed 00000001:00080000:1.0:1787784666.478711:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967332 is committed 00000100:00100000:1.0:1787784666.484680:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.484826:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657998080 00000100:00100000:2.0:1787784666.484837:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657998080 00000100:00100000:2.0:1787784666.484894:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 726 00000100:00100000:2.0:1787784666.484903:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657998080:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784666.484955:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784666.484980:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4ade lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0930 expref: 34 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.484999:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784666.485005:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784666.485036:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.485050:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.485060:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784666.485289:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a226d4b5 reply state 000000006780f875 transno 4294967765 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.485299:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784666.485458:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:21871:x1874627657998080:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 554us (788us total) trans 4294967765 rc 0/0 00000100:00100000:2.0:1787784666.485467:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 726 00010000:00010000:1.0:1787784666.486860:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.486875:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.486881:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.486887:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784666.486892:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.486900:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.486928:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:1.0:1787784666.486931:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784666.486935:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 2/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.486942:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea800/0xb0ce63137acc4afa lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784666.488188:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.488304:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657998208 00000100:00100000:2.0:1787784666.488314:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657998208 00000100:00100000:2.0:1787784666.488363:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 186 00000100:00100000:2.0:1787784666.488373:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627657998208:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1787784666.488632:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:2.0:1787784666.488637:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb mtime from disk: 1787784665 -> 1787784661 00002000:00010000:2.0:1787784666.488639:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb atime from disk: 0 -> 1787784661 00002000:00010000:2.0:1787784666.488641:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb ctime from disk: 1787784665 -> 1787784664 00002000:00010000:2.0:1787784666.488643:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1787784666.488702:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627657998208:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 330us (559us total) trans 4294967334 rc 0/0 00000100:00100000:2.0:1787784666.488710:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 186 00000100:00100000:2.0:1787784666.494016:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.494098:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657998464 00000100:00100000:3.0:1787784666.494109:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657998464 00000100:00100000:1.0:1787784666.494279:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 727 00000100:00100000:1.0:1787784666.494291:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+34:21871:x1874627657998464:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1787784666.494363:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.494381:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 34 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.494392:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.494438:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 34 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.494453:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 34 pid: 5813 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784666.494467:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a91e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668338048:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784666.494553:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1540 going to sleep for 11 seconds 00000100:00100000:0.0:1787784666.498291:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a91e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668338048:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784666.498321:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784666.502638:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.502864:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657998592 00000100:00100000:3.0:1787784666.502873:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657998592 00010000:00010000:3.0:1787784666.502914:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 35 pid: 5813 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784666.502957:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 199 00000100:00100000:3.0:1787784666.502967:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ec450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:2357:x1874627657998592:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784666.502980:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784666.502986:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 35 pid: 5813 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.503000:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 35 pid: 5813 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.503021:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4ac2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0922 expref: 35 pid: 5813 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.503052:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:3.0:1787784666.503056:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.503065:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.503071:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.503102:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784666.503188:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ec450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2357:x1874627657998592:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 222us (559us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784666.503196:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 199 00010000:00010000:0.0:1787784666.503296:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503305:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503335:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503343:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503348:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503354:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784666.503359:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503377:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503387:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784666.503390:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787784666.503394:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.503400:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc4b01 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784666.503455:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21871:x1874627657998464:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 9166us (9447us total) trans 0 rc -22/-22 00000100:00100000:0.0:1787784666.503464:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 727 00000100:00100000:1.0:1787784666.517721:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.517857:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657998720 00000100:00100000:2.0:1787784666.517866:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657998720 00000100:00100000:2.0:1787784666.517922:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:2.0:1787784666.517933:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627657998720:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784666.517977:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.517998:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.518007:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784666.518012:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784666.518016:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.518052:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.518086:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 11 pid: 6671 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.518101:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953937b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668338176:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784666.518463:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784666.523448:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953937b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668338176:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784666.523480:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784666.524522:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.524744:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657998848 00000100:00100000:3.0:1787784666.524753:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657998848 00010000:00010000:3.0:1787784666.524870:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 12 pid: 6671 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784666.524915:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 200 00000100:00100000:3.0:1787784666.524926:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627657998848:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784666.524943:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784666.524964:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 12 pid: 6671 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.524978:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 12 pid: 6671 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.525016:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000036d766d5/0xb0ce63137acc4aec lrc: 0/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b093e expref: 12 pid: 6671 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.525033:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784666.525037:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784666.525043:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.525053:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.525091:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784666.525196:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627657998848:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 223us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784666.525205:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 200 00010000:00010000:2.0:1787784666.525329:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525339:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525363:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525372:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525380:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525399:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784666.525404:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525413:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525443:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.525451:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d69e17df/0xb0ce63137acc4b08 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.525496:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627657998720:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 7566us (7784us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.525505:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:1.0:1787784666.527714:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.536519:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657999104 00000100:00100000:2.0:1787784666.536534:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657999104 00000100:00100000:2.0:1787784666.536649:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 728 00000100:00100000:2.0:1787784666.536660:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+33:21871:x1874627657999104:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784666.536740:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.536758:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.536802:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.537025:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.537036:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 34 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.537044:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.537077:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 34 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.537092:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 34 pid: 5812 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.537106:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953937b840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668338304:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784666.537219:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784666.541655:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953937b840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668338304:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784666.541687:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784666.543576:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.544281:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657999232 00000100:00100000:3.0:1787784666.544292:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657999232 00010000:00010000:2.0:1787784666.544485:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 35 pid: 5812 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784666.544561:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 201 00000100:00100000:2.0:1787784666.544584:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:2357:x1874627657999232:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784666.544605:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784666.544611:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 35 pid: 5812 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.544624:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 35 pid: 5812 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.544644:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4ad0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0929 expref: 35 pid: 5812 timeout: 557 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.544661:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784666.544667:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.544675:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.544682:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784666.544716:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784666.544861:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2357:x1874627657999232:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 278us (1292us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.544870:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 201 00010000:00010000:1.0:1787784666.545176:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784666.545186:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784666.545343:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967766 00000001:00080000:1.0:1787784666.545503:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967767 00000001:00080000:1.0:1787784666.545544:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967768 00000100:00100000:1.0:1787784666.545617:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668338432:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787784666.545668:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784666.545700:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1000 going to sleep for 11 seconds 00000100:00100000:0.0:1787784666.546094:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668338432 00000100:00100000:0.0:1787784666.546101:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668338432 00000100:00100000:0.0:1787784666.546169:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23 00000100:00100000:0.0:1787784666.546179:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668338432:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784666.546255:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668338432:12345-0@lo:1000:mdt00_003.0 Request processed in 77us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.546263:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23 00000100:00100000:1.0:1787784666.546628:0:10850:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967333 gen 2 00000004:00080000:1.0:1787784666.546634:0:10850:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff98951b536580, transno 4294967333 00000100:00100000:1.0:1787784666.546648:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668338432:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1787784666.547409:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967769 00000100:00100000:1.0:1787784666.547910:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668338560:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787784666.547961:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784666.547992:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1540 going to sleep for 11 seconds 00000100:00100000:0.0:1787784666.548263:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668338560 00000100:00100000:0.0:1787784666.548270:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668338560 00000100:00100000:0.0:1787784666.548314:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24 00000100:00100000:0.0:1787784666.548322:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668338560:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784666.548510:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:10850:x1874627668338560:12345-0@lo:1000:mdt00_003.0 Request processed in 188us (548us total) trans 4294967335 rc 0/0 00000100:00100000:0.0:1787784666.548532:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24 00000100:00100000:3.0:1787784666.548681:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668338560:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787784666.548716:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548726:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548744:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548761:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784666.548765:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548782:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548798:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989503a6de00) 00010000:00010000:3.0:1787784666.548801:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784666.548805:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548811:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb200/0xb0ce63137acc4b16 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548838:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548845:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548851:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548856:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbcc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784666.548859:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548864:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548882:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6dd00) 00010000:00010000:3.0:1787784666.548884:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784666.548891:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b090d expref: 72 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548901:0:10850:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b090d expref: 72 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548910:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b090d expref: 72 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.548938:0:10850:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894010aca00/0xb0ce63137acc4a67 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b090d expref: 72 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784666.548950:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668338688:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784666.549025:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784666.551182:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668338688:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784666.551208:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551217:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbcc00/0xb0ce63137acc4a60 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551242:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551254:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551274:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551283:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551289:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551294:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551299:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784666.551302:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551308:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551317:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6dd00) 00010000:00010000:3.0:1787784666.551320:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784666.551323:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551328:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4b1d lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551337:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551342:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551352:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551356:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784666.551359:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551364:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551370:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff989503a6dd00) 00010000:00010000:3.0:1787784666.551372:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784666.551374:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.551379:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4b0f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784666.551589:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:21871:x1874627657999104:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 14930us (23882us total) trans 4294967770 rc 0/0 00000100:00100000:3.0:1787784666.551599:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 728 00000020:00080000:3.0:1787784666.551604:0:10850:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:0.0:1787784666.552108:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784666.552113:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784666.552682:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784666.552691:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784666.553251:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784666.553446:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657999360 00000100:00100000:2.0:1787784666.553454:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657999360 00000100:00100000:2.0:1787784666.553529:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 147 00000100:00100000:2.0:1787784666.553539:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21870:x1874627657999360:12345-192.168.203.43@tcp:49:md5sum.0 00000100:00100000:2.0:1787784666.553677:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21870:x1874627657999360:12345-192.168.203.43@tcp:49:md5sum.0 Request processed in 140us (428us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.553685:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 147 00000100:00100000:2.0:1787784666.554272:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:1.0:1787784666.554563:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784666.555389:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657999616 00000100:00100000:3.0:1787784666.555398:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657999616 00000100:00100000:3.0:1787784666.555549:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 148 00000100:00100000:3.0:1787784666.555560:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:21871:x1874627657999616:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1787784666.558598:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x13:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1787784666.558749:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967771 00010000:00010000:3.0:1787784666.558834:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.558850:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.558862:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.558871:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.558877:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.558884:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.558901:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784666.559411:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:21871:x1874627657999616:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 3848us (5134us total) trans 4294967772 rc 0/0 00000100:00100000:3.0:1787784666.559432:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 148 00000020:00080000:3.0:1787784666.559438:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1787784666.559464:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784666.559470:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.559477:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784666.559481:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.559488:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.559508:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff989505542200) 00010000:00010000:3.0:1787784666.559511:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784666.559516:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.559522:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4b24 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784666.559539:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784666.559541:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784666.559543:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784666.559670:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784666.560112:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657999744 00000100:00100000:0.0:1787784666.560120:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657999744 00000100:00100000:0.0:1787784666.560326:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 188 00000100:00100000:0.0:1787784666.560336:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+9:21870:x1874627657999744:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:0.0:1787784666.560350:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784666.560366:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008fb8937/0xb0ce63137acc4b2b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0945 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784666.560391:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x10:0x0] initial LVB size: 158, mtime: 0x6a8f6dd5, atime: 0x6a8f6dd5, ctime: 0x6a8f6dd8, blocks: 0xb 00010000:00010000:0.0:1787784666.560409:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=457s v=(0 0) 00010000:00010000:0.0:1787784666.560518:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784666.560530:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008fb8937/0xb0ce63137acc4b2b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0945 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.560546:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008fb8937/0xb0ce63137acc4b2b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0945 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784666.560560:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbda00, rc 0) 00000100:00100000:0.0:1787784666.560609:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:21870:x1874627657999744:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 274us (940us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.560617:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 188 00000100:00080000:1.0:1787784666.561102:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784666.563407:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.563493:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627657999872 00000100:00100000:3.0:1787784666.563503:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627657999872 00002000:00100000:3.0:1787784666.563549:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x10:0x0] (0->4095) req@ffff989539378000 x1874627657999872/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:547/0 lens 488/0 e 0 to 0 dl 1787784677 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784666.563561:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539378000 with x1874627657999872 ext(0->4095) 00002000:00010000:3.0:1787784666.563566:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539378000 00000100:00100000:3.0:1787784666.563600:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 59 00000100:00100000:3.0:1787784666.563610:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539378000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627657999872:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:3.0:1787784666.563857:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539378000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627657999872:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 247us (457us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784666.563865:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 59 00002000:00100000:3.0:1787784666.563872:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x10:0x0] (0->4095) req@ffff989539378000 x1874627657999872/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:547/0 lens 488/4536 e 0 to 0 dl 1787784677 ref 1 fl Complete:/600/0 rc 4096/4096 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784666.563880:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539378000 with x1874627657999872 ext(0->4095) 00002000:00010000:3.0:1787784666.563883:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539378000 00000100:00100000:2.0:1787784666.569974:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784666.570037:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658000128 00000100:00100000:3.0:1787784666.570047:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658000128 00000100:00100000:3.0:1787784666.570139:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 149 00000100:00100000:3.0:1787784666.570177:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950767ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:21870:x1874627658000128:12345-192.168.203.43@tcp:35:md5sum.0 00000100:00100000:3.0:1787784666.570505:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950767ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:21870:x1874627658000128:12345-192.168.203.43@tcp:35:md5sum.0 Request processed in 357us (656us total) trans 4294967773 rc 0/0 00000100:00100000:3.0:1787784666.570513:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 149 00000100:00100000:0.0:1787784666.609475:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668339072:0@lo:400:kworker.0 00000100:00100000:3.0:1787784666.609501:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668338944:0@lo:400:kworker.0 00000100:00100000:0.0:1787784666.609544:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784666.609589:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339072 00000100:00100000:3.0:1787784666.609597:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339072 00000100:00100000:0.0:1787784666.609602:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784666.609637:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668338944 00000100:00100000:0.0:1787784666.609641:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668338944 00000100:00100000:3.0:1787784666.609657:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:2.0:1787784666.609729:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668338816:0@lo:400:kworker.0 00000100:00100000:2.0:1787784666.609763:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784666.609793:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784666.609799:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539377c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668339200:0@lo:400:kworker.0 00000100:00100000:2.0:1787784666.609824:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784666.609854:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668338816 00000100:00100000:2.0:1787784666.609858:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668338816 00000100:00100000:2.0:1787784666.609886:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:2.0:1787784666.609894:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668338816:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1787784666.609912:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784666.609943:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668338816:12345-0@lo:400:kworker.0 Request processed in 50us (178us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.609950:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:2.0:1787784666.609993:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668338816:0@lo:400:kworker.0 00000100:00100000:2.0:1787784666.610024:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339200 00000100:00100000:2.0:1787784666.610029:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339200 00000100:00100000:3.0:1787784666.610057:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668339072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784666.610108:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668339072:12345-0@lo:400:kworker.0 Request processed in 443us (565us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784666.610180:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668339072:0@lo:400:kworker.0 00000100:00100000:0.0:1787784666.610262:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:0.0:1787784666.610271:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668338944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784666.610407:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668338944:12345-0@lo:400:kworker.0 Request processed in 137us (805us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.610414:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:0.0:1787784666.610481:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668339328:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784666.610507:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784666.610538:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339328 00000100:00100000:0.0:1787784666.610542:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339328 00000100:00100000:0.0:1787784666.610573:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:0.0:1787784666.610582:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668339328:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784666.610594:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00002000:00000024:0.0:1787784666.610597:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3854565376 00000100:00100000:2.0:1787784666.610604:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668339200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784666.610640:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668339328:12345-0@lo:13:osp-pre-1-0.0 Request processed in 60us (131us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784666.610651:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00100000:2.0:1787784666.610652:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668339200:12345-0@lo:400:kworker.0 Request processed in 47us (823us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784666.610672:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:0.0:1787784666.610689:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668339328:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784666.610701:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539377c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668339200:0@lo:400:kworker.0 00000100:00080000:2.0:1787784666.610716:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784666.615136:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:1.0:1787784666.616483:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668338944:0@lo:400:kworker.0 00010000:00010000:2.0:1787784667.058624:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787784667.715090:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967326 is committed 00000001:00080000:2.0:1787784667.715104:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967325 is committed 00010000:00010000:3.0:1787784668.081892:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784669.107876:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784669.684064:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668339456:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784669.684143:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784669.684285:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339456 00000100:00100000:3.0:1787784669.684293:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339456 00000100:00100000:3.0:1787784669.684387:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:3.0:1787784669.684411:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668339456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784669.684448:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 61276160 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784669.684495:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668339456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 100us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784669.684502:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:3.0:1787784669.684540:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967326 gen 2 00000004:00080000:3.0:1787784669.684560:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989539379680, transno 4294967326 00000100:00100000:3.0:1787784669.684577:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668339456:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787784669.684964:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784669.966377:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784669.966474:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784669.966501:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784669.966527:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784669.966541:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784669.966557:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784669.966579:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784669.966621:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658000512 00000100:00100000:1.0:1787784669.966630:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658000512 00000100:00100000:1.0:1787784669.966697:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658001024 00000100:00100000:1.0:1787784669.966700:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658001024 00000100:00100000:1.0:1787784669.966714:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658000640 00000100:00100000:1.0:1787784669.966716:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658000640 00000100:00100000:1.0:1787784669.966726:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658000896 00000100:00100000:1.0:1787784669.966728:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658000896 00000100:00100000:1.0:1787784669.966743:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 223 00000100:00100000:1.0:1787784669.966751:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527deac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658000512:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784669.966819:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527deac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658000512:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 68us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784669.966826:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 223 00000100:00100000:1.0:1787784669.966981:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 224 00000100:00100000:1.0:1787784669.966987:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658001024:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784669.967023:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658001024:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (496us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784669.967029:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 224 00000100:00100000:1.0:1787784669.967039:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 225 00000100:00100000:1.0:1787784669.967044:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658000640:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784669.967059:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658000640:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 15us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784669.967064:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 225 00000100:00100000:1.0:1787784669.967071:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 226 00000100:00100000:1.0:1787784669.967075:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658000896:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784669.967089:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658000896:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 14us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784669.967094:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 226 00000100:00100000:1.0:1787784669.967108:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658000768 00000100:00100000:1.0:1787784669.967111:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658000768 00000100:00100000:1.0:1787784669.967131:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658000384 00000100:00100000:1.0:1787784669.967133:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658000384 00000100:00100000:1.0:1787784669.967166:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00100000:1.0:1787784669.967172:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2356:x1874627658000768:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784669.967198:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2356:x1874627658000768:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (696us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784669.967204:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00100000:1.0:1787784669.967213:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 281 00000100:00100000:1.0:1787784669.967217:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2356:x1874627658000384:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784669.967250:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2356:x1874627658000384:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (708us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784669.967258:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 281 00000100:00100000:2.0:1787784669.967262:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658000256 00000100:00100000:2.0:1787784669.967268:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658000256 00000100:00080000:1.0:1787784669.967273:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784669.967278:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784669.967280:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784669.967375:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 351 00000100:00100000:2.0:1787784669.967385:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658000256:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784669.967427:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658000256:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 45us (1061us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784669.967434:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 351 00010000:00010000:3.0:1787784670.129541:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784671.153371:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784671.500139:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784671.500297:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658001152 00000100:00100000:3.0:1787784671.500307:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658001152 00002000:00100000:3.0:1787784671.500367:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xd:0x0] (0->157) req@ffff989402a90780 x1874627658001152/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:552/0 lens 648/0 e 0 to 0 dl 1787784682 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784671.500378:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402a90780 with x1874627658001152 ext(0->157) 00002000:00010000:3.0:1787784671.500393:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402a90780 00000100:00100000:3.0:1787784671.500422:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 60 00000100:00100000:3.0:1787784671.500432:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658001152:12345-192.168.203.43@tcp:4:cp.0 00000100:00080000:1.0:1787784671.500817:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784671.501063:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658001152:12345-192.168.203.43@tcp:4:cp.0 Request processed in 631us (931us total) trans 4294967327 rc 0/0 00000100:00100000:3.0:1787784671.501071:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 60 00002000:00100000:3.0:1787784671.501079:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xd:0x0] (0->157) req@ffff989402a90780 x1874627658001152/t4294967327(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:552/0 lens 648/448 e 0 to 0 dl 1787784682 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784671.501087:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402a90780 with x1874627658001152 ext(0->157) 00002000:00010000:3.0:1787784671.501092:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402a90780 00000100:00100000:0.0:1787784671.729736:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784671.729765:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668339584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784671.729845:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784671.730676:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339584 00000100:00100000:3.0:1787784671.730687:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339584 00000100:00100000:3.0:1787784671.730745:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:3.0:1787784671.730755:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668339584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784671.730791:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784671.730860:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668339584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (1015us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784671.730876:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:3.0:1787784671.730946:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668339584:0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1787784671.735443:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784671.749860:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668339712:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.749931:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784671.750001:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339712 00000100:00100000:1.0:1787784671.750009:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339712 00000100:00100000:1.0:1787784671.750053:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1787784671.750062:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668339712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.750114:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668339712:12345-0@lo:400:kworker.0 Request processed in 53us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784671.750121:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1787784671.750190:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668339712:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.750229:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668339840:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.750260:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784671.750295:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339840 00000100:00100000:1.0:1787784671.750299:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339840 00000100:00100000:1.0:1787784671.750334:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1787784671.750342:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668339840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.750618:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668339840:12345-0@lo:400:kworker.0 Request processed in 39us (120us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784671.750639:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1787784671.750671:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668339840:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.750778:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668339968:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.750906:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784671.750961:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668340096:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.750987:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784671.751016:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668340096 00000100:00100000:1.0:1787784671.751020:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668340096 00000100:00100000:1.0:1787784671.751054:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1787784671.751061:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668340096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.751097:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668340096:12345-0@lo:400:kworker.0 Request processed in 37us (109us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784671.751104:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00080000:1.0:1787784671.751120:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784671.751141:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668340096:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.751203:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668340224:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.751229:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784671.751671:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668339968 00000100:00100000:0.0:1787784671.751676:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668339968 00000100:00100000:0.0:1787784671.751779:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668340224 00000100:00100000:0.0:1787784671.751783:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668340224 00000100:00100000:0.0:1787784671.751798:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1787784671.751805:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668339968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784671.751917:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668339968:12345-0@lo:400:kworker.0 Request processed in 112us (1010us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784671.751923:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1787784671.751941:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:0.0:1787784671.751947:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668340224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784671.751975:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668340224:12345-0@lo:400:kworker.0 Request processed in 29us (746us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784671.751980:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00080000:0.0:1787784671.768185:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784671.768900:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668339968:0@lo:400:kworker.0 00000100:00100000:1.0:1787784671.768948:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668340224:0@lo:400:kworker.0 00000001:00080000:0.0:1787784671.808638:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967770 is committed 00000020:00080000:0.0:1787784671.808652:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989507fd1000 refcount 2 committed 0 result 0 batchid 10 00000020:00080000:0.0:1787784671.808658:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953f510d00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989507fd1c00 00000020:00080000:0.0:1787784671.808662:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953f510500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895393c0800 00000001:00080000:0.0:1787784671.808689:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967773 is committed 00000001:00080000:0.0:1787784671.808694:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967772 is committed 00000020:00080000:0.0:1787784671.808705:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 10 committed batchid 9 00000020:00080000:0.0:1787784671.808881:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 9 00000001:00080000:1.0:1787784671.818284:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967335 is committed 00000001:00080000:1.0:1787784671.818301:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967334 is committed 00000100:00100000:1.0:1787784671.839774:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784671.839994:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658001280 00000100:00100000:0.0:1787784671.840005:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658001280 00000100:00100000:0.0:1787784671.840088:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 729 00000100:00100000:0.0:1787784671.840100:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21818:x1874627658001280:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1787784671.840243:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21818:x1874627658001280:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 145us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784671.840251:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 729 00000100:00080000:2.0:1787784671.840321:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784671.846623:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784671.846723:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658001536 00000100:00100000:2.0:1787784671.846734:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658001536 00000100:00100000:2.0:1787784671.846793:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 150 00000100:00100000:2.0:1787784671.846803:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:21818:x1874627658001536:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784671.847321:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+32:21818:x1874627658001536:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 518us (707us total) trans 4294967774 rc 0/0 00000100:00100000:2.0:1787784671.847329:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 150 00010000:00010000:3.0:1787784672.177509:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784672.689449:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784672.689482:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784672.689530:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 18 count 3 00000040:00080000:1.0:1787784672.689537:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:1.0:1787784672.859860:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967327 is committed 00010000:00010000:3.0:1787784673.201584:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787784674.203340:0:17400:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 18:51:11 (1787784671) 00010000:00010000:3.0:1787784674.227138:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784674.802531:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668340352:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784674.802599:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784674.802719:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668340352 00000100:00100000:0.0:1787784674.802727:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668340352 00000100:00100000:0.0:1787784674.802788:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:0.0:1787784674.802797:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668340352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784674.802831:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784674.802876:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668340352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784674.802882:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:0.0:1787784674.802932:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668340352:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787784674.802986:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784675.082411:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.082451:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.082471:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.082497:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.082518:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.082536:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.082573:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.082603:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002048 00000100:00100000:1.0:1787784675.082612:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002048 00000100:00100000:1.0:1787784675.082665:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002304 00000100:00100000:1.0:1787784675.082668:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002304 00000100:00100000:1.0:1787784675.082683:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658001920 00000100:00100000:1.0:1787784675.082686:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658001920 00000100:00100000:1.0:1787784675.082698:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002432 00000100:00100000:1.0:1787784675.082700:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002432 00000100:00100000:1.0:1787784675.082715:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 228 00000100:00100000:1.0:1787784675.082723:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658002048:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784675.082790:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658002048:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 67us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.082798:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 228 00000100:00100000:1.0:1787784675.082823:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 229 00000100:00100000:1.0:1787784675.082829:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658002304:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784675.082858:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658002304:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (385us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.082865:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 229 00000100:00100000:1.0:1787784675.082877:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:1.0:1787784675.082883:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658001920:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784675.082904:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658001920:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.082910:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:1.0:1787784675.082920:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 231 00000100:00100000:1.0:1787784675.082926:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658002432:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784675.082950:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658002432:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.082955:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 231 00000100:00100000:1.0:1787784675.082977:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658001664 00000100:00100000:1.0:1787784675.082981:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658001664 00000100:00100000:1.0:1787784675.083007:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 353 00000100:00100000:1.0:1787784675.083014:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658001664:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784675.083038:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658001664:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (540us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.083044:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 353 00000100:00100000:1.0:1787784675.083074:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658001792 00000100:00100000:1.0:1787784675.083077:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658001792 00000100:00100000:1.0:1787784675.083107:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002176 00000100:00100000:1.0:1787784675.083110:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002176 00000100:00100000:1.0:1787784675.083136:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:1.0:1787784675.083142:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:2356:x1874627658001792:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784675.083195:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:2356:x1874627658001792:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 52us (793us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.083201:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:1.0:1787784675.083213:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:1.0:1787784675.083219:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+32:2356:x1874627658002176:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784675.084348:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+32:2356:x1874627658002176:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1121us (1803us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.084367:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00080000:1.0:1787784675.084546:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784675.084551:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784675.084554:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787784675.249367:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1787784675.447492:0:17448:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007fffd31fe440 20000000:00000080:2.0:1787784675.447510:0:17448:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff989401d17400 uarg=0000000000000000 00000040:00100000:2.0:1787784675.449828:0:17448:0:(llog.c:1072:llog_reverse_process()) index 41 : idx 1 00000040:00080000:2.0:1787784675.450489:0:17448:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].42, 224 off15448 20000000:01000000:2.0:1787784675.450510:0:17448:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce020 lustre test_11c (null) (null) 00000040:00080000:2.0:1787784675.450652:0:17448:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].43, 112 off15560 00000040:00080000:2.0:1787784675.450684:0:17448:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].44, 224 off15784 00000040:00080000:2.0:1787784675.451123:0:17448:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].37, 224 off14712 20000000:01000000:2.0:1787784675.451135:0:17448:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce020 lustre test_11c (null) (null) 00000040:00080000:2.0:1787784675.451408:0:17448:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].38, 120 off14832 00000040:00080000:2.0:1787784675.451458:0:17448:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].39, 224 off15056 10000000:01000000:2.0:1787784675.451473:0:17448:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:2.0:1787784675.451491:0:17448:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.451508:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff98953efb3400/0xb0ce63137acc1c4d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba0 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453693:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff98953efb2000/0xb0ce63137acc1c54 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba7 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453710:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989539135600/0xb0ce63137acc1c62 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc1c5b expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453718:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989500031000/0xb0ce63137acc1c70 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc1c69 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453725:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989539135e00/0xb0ce63137acc1c7e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc1c77 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453731:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989500031a00/0xb0ce63137acc1c8c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc1c85 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453738:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989500030600/0xb0ce63137acc1c9a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc1c93 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453744:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989500030c00/0xb0ce63137acc1ca8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc1ca1 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453770:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989500030a00/0xb0ce63137acc1cb6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc1caf expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453778:0:17448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff98953efb2a00/0xb0ce63137acc1cf5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afbb5 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.453786:0:17448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.454314:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff98953efb2a00/0xb0ce63137acc1cf5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afbb5 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.454330:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff98953efb2a00/0xb0ce63137acc1cf5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afbb5 expref: 12 pid: 5804 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.454347:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340480:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784675.454493:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989500030a00/0xb0ce63137acc1cb6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc1caf expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.454501:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989500030a00/0xb0ce63137acc1cb6 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc1caf expref: 18 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.454511:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340608:0@lo:104: 00000100:00100000:2.0:1787784675.454643:0:17448:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784675.454708:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989500030c00/0xb0ce63137acc1ca8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc1ca1 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.454717:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989500030c00/0xb0ce63137acc1ca8 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc1ca1 expref: 18 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.454733:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340736:0@lo:104: 00000100:00100000:2.0:1787784675.454753:0:17448:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784675.455485:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989500030600/0xb0ce63137acc1c9a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc1c93 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.455495:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989500030600/0xb0ce63137acc1c9a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc1c93 expref: 18 pid: 5805 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.455505:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6ab840 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340864:0@lo:104: 00000100:00100000:2.0:1787784675.455553:0:17448:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784675.455758:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668340608 00000100:00100000:0.0:1787784675.455766:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668340608 00000100:00100000:0.0:1787784675.455814:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668340736 00000100:00100000:0.0:1787784675.455817:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668340736 00000100:00100000:0.0:1787784675.455831:0:5793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 28 00000100:00100000:0.0:1787784675.455839:0:5793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+4:17448:x1874627668340608:12345-0@lo:104: 00010000:00010000:0.0:1787784675.455851:0:5793:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.455912:0:5793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787784675.455918:0:5793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+4:17448:x1874627668340608:12345-0@lo:104: Request processed in 80us (1276us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784675.455925:0:5793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 28 00000100:00100000:0.0:1787784675.455946:0:5793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 29 00000100:00100000:0.0:1787784675.455952:0:5793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668340736:12345-0@lo:104: 00010000:00010000:0.0:1787784675.455968:0:5793:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456002:0:5793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:0.0:1787784675.456008:0:5793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668340736:12345-0@lo:104: Request processed in 56us (1253us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784675.456014:0:5793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 29 00010000:00010000:0.0:1787784675.456032:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787784675.456038:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456045:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953efb2200 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784675.456050:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456060:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784675.456068:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784675.456073:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784675.456099:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456106:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784675.456123:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456130:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98953efb2200/0xb0ce63137acc1caf lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1cb6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456164:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784675.456168:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456174:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a114e00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784675.456176:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456183:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784675.456191:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784675.456196:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784675.456207:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456212:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784675.456218:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456224:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98952a114e00/0xb0ce63137acc1ca1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1ca8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.456232:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784675.456233:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784675.456235:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784675.456253:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784675.456277:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668340992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784675.456307:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784675.456330:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784675.456336:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668341120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784675.456548:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784675.456747:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668340992 00000100:00100000:3.0:1787784675.456753:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668340992 00010000:00010000:3.0:1787784675.456774:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989500030a00/0xb0ce63137acc1cb6 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1caf expref: 19 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.456795:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668341120 00000100:00100000:3.0:1787784675.456798:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668341120 00010000:00010000:3.0:1787784675.456808:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989500030c00/0xb0ce63137acc1ca8 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1ca1 expref: 20 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.456824:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1787784675.456830:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+20:3276:x1874627668340992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784675.456841:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784675.456846:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989500030a00/0xb0ce63137acc1cb6 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1caf expref: 20 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.456855:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989500030a00/0xb0ce63137acc1cb6 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc1caf expref: 20 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.456873:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784675.456922:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+20:3276:x1874627668340992:12345-0@lo:103:ldlm_bl.0 Request processed in 92us (614us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784675.456941:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1787784675.456955:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1787784675.456961:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668341120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784675.456969:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784675.456973:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989500030c00/0xb0ce63137acc1ca8 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1ca1 expref: 19 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.456981:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989500030c00/0xb0ce63137acc1ca8 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc1ca1 expref: 19 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.456992:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784675.457014:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668341120:12345-0@lo:103:ldlm_bl.0 Request processed in 53us (466us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784675.457021:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1787784675.457124:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668341120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784675.457132:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668340992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784675.457429:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668340864 00000100:00100000:0.0:1787784675.457435:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668340864 00000100:00100000:0.0:1787784675.457466:0:5793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30 00000100:00100000:0.0:1787784675.457472:0:5793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668340864:12345-0@lo:104: 00010000:00010000:0.0:1787784675.457481:0:5793:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.457516:0:5793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787784675.457521:0:5793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668340864:12345-0@lo:104: Request processed in 49us (1968us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784675.457526:0:5793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30 00010000:00010000:0.0:1787784675.459693:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784675.459703:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.459713:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953efb3000 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784675.459717:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.459725:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784675.459743:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784675.459750:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784675.459798:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.459805:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784675.459827:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.459834:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98953efb3000/0xb0ce63137acc1c93 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.459846:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784675.459848:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784675.459850:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784675.459863:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668341248:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784675.459975:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784675.460020:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668341248 00000100:00100000:0.0:1787784675.460026:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668341248 00010000:00010000:0.0:1787784675.460047:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989500030600/0xb0ce63137acc1c9a lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c93 expref: 19 pid: 5805 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784675.460072:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1787784675.460080:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668341248:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787784675.460091:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784675.460096:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989500030600/0xb0ce63137acc1c9a lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c93 expref: 19 pid: 5805 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.460111:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989500030600/0xb0ce63137acc1c9a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc1c93 expref: 19 pid: 5805 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.460128:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784675.460181:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668341248:12345-0@lo:103:ldlm_bl.0 Request processed in 101us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784675.460189:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1787784675.460226:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668341248:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784675.462744:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784675.462800:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002560 00000100:00100000:3.0:1787784675.462809:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002560 00010000:00010000:3.0:1787784675.462842:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff98953efb2a00/0xb0ce63137acc1cf5 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afbb5 expref: 13 pid: 5804 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.462876:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 205 00000100:00100000:3.0:1787784675.462885:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658002560:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784675.463083:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989500031a00/0xb0ce63137acc1c8c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc1c85 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.463097:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989500031a00/0xb0ce63137acc1c8c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc1c85 expref: 18 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.463111:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028da40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668341376:0@lo:104: 00000100:00100000:2.0:1787784675.463194:0:17448:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784675.463233:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989539135e00/0xb0ce63137acc1c7e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc1c77 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.463438:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784675.463447:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff98953efb2a00/0xb0ce63137acc1cf5 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afbb5 expref: 13 pid: 5804 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.463478:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989539135e00/0xb0ce63137acc1c7e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc1c77 expref: 18 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.463488:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668341504:0@lo:104: 00000100:00100000:2.0:1787784675.463515:0:17448:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784675.463584:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668341376 00000100:00100000:1.0:1787784675.463590:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668341376 00000100:00100000:1.0:1787784675.463619:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668341504 00000100:00100000:1.0:1787784675.463621:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668341504 00000100:00100000:1.0:1787784675.463634:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 31 00000100:00100000:1.0:1787784675.463641:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17448:x1874627668341376:12345-0@lo:104: 00010000:00010000:1.0:1787784675.463654:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784675.463696:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787784675.463703:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17448:x1874627668341376:12345-0@lo:104: Request processed in 62us (509us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.463709:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 31 00000100:00100000:1.0:1787784675.463739:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 32 00000100:00100000:1.0:1787784675.463745:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668341504:12345-0@lo:104: 00010000:00010000:1.0:1787784675.463753:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784675.463783:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:1.0:1787784675.463788:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668341504:12345-0@lo:104: Request processed in 44us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784675.463793:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 32 00010000:00010000:0.0:1787784675.465473:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787784675.465484:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465494:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784675.465498:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465509:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784675.465516:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784675.465522:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784675.465569:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465575:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784675.465606:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465630:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989500030e00/0xb0ce63137acc1c85 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465645:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784675.465649:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465655:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115a00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784675.465658:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465666:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784675.465672:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784675.465678:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784675.465693:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465699:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784675.465713:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465721:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98952a115a00/0xb0ce63137acc1c77 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c7e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.465730:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784675.465736:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784675.465738:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784675.465746:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784675.465763:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668341760:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784675.465815:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784675.465862:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668341760 00000100:00100000:0.0:1787784675.465868:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668341760 00010000:00010000:0.0:1787784675.465890:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989539135e00/0xb0ce63137acc1c7e lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c77 expref: 19 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784675.465914:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:0.0:1787784675.465923:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ed850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668341760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787784675.465937:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784675.465942:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989539135e00/0xb0ce63137acc1c7e lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c77 expref: 19 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784675.466007:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668341632:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784675.466037:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787784675.466371:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784675.467800:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff98953efb2a00/0xb0ce63137acc1cf5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afbb5 expref: 13 pid: 5804 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.467843:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784675.467916:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658002560:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5030us (5177us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784675.467926:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 205 00000100:00100000:3.0:1787784675.467952:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668341632 00000100:00100000:3.0:1787784675.467957:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668341632 00010000:00010000:3.0:1787784675.467978:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989500031a00/0xb0ce63137acc1c8c lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c85 expref: 20 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.467996:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1787784675.468004:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+20:3279:x1874627668341632:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784675.468016:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784675.468021:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989500031a00/0xb0ce63137acc1c8c lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c85 expref: 20 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.468030:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989500031a00/0xb0ce63137acc1c8c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc1c85 expref: 20 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.468040:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784675.468088:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+20:3279:x1874627668341632:12345-0@lo:103:ldlm_bl.0 Request processed in 85us (2050us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784675.468094:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1787784675.468166:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668341632:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784675.469479:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989500031000/0xb0ce63137acc1c70 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc1c69 expref: 19 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.469494:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989500031000/0xb0ce63137acc1c70 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc1c69 expref: 19 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.469518:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ad2c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668341888:0@lo:104: 00000100:00100000:3.0:1787784675.469581:0:17448:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784675.469627:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989539135600/0xb0ce63137acc1c62 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc1c5b expref: 19 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.469637:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989539135600/0xb0ce63137acc1c62 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc1c5b expref: 19 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.469646:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ade00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668342016:0@lo:104: 00000100:00100000:3.0:1787784675.469674:0:17448:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784675.469711:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff98953efb2000/0xb0ce63137acc1c54 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba7 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.469721:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff98953efb2000/0xb0ce63137acc1c54 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba7 expref: 12 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.469732:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668342144:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784675.469773:0:17448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff98953efb3400/0xb0ce63137acc1c4d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba0 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784675.469782:0:17448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff98953efb3400/0xb0ce63137acc1c4d lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba0 expref: 12 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.469792:0:17448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668342272:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784675.469814:0:17448:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1780 going to sleep for 11 seconds 00010000:00010000:3.0:1787784675.469831:0:17448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff98953efb2a00/0xb0ce63137acc1cf5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afbb5 expref: 12 pid: 5804 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.469853:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340480:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784675.469869:0:17448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989500030a00/0xb0ce63137acc1cb6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc1caf expref: 19 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.469877:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340608:0@lo:104: 00010000:00010000:3.0:1787784675.469887:0:17448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989500030c00/0xb0ce63137acc1ca8 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc1ca1 expref: 18 pid: 5804 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.469895:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340736:0@lo:104: 00000100:00100000:2.0:1787784675.470081:0:5793:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787784675.470094:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668341888 00000100:00100000:2.0:1787784675.470105:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668341888 00000100:00100000:2.0:1787784675.470144:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668342016 00000100:00100000:2.0:1787784675.470173:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668342016 00000100:00100000:2.0:1787784675.470190:0:5793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 33 00000100:00100000:2.0:1787784675.470198:0:5793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+4:17448:x1874627668341888:12345-0@lo:104: 00010000:00010000:2.0:1787784675.470210:0:5793:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470268:0:5793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787784675.470275:0:5793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+4:17448:x1874627668341888:12345-0@lo:104: Request processed in 78us (693us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784675.470282:0:5793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 33 00000100:00100000:2.0:1787784675.470298:0:5793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1787784675.470304:0:5793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895401a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668342016:12345-0@lo:104: 00010000:00010000:2.0:1787784675.470311:0:5793:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470344:0:5793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1787784675.470349:0:5793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895401a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+3:17448:x1874627668342016:12345-0@lo:104: Request processed in 46us (675us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784675.470372:0:5793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00010000:00010000:2.0:1787784675.470411:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787784675.470416:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470423:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a114200 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:2.0:1787784675.470427:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470446:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787784675.470454:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784675.470459:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787784675.470491:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470498:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787784675.470516:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470523:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98952a114200/0xb0ce63137acc1c69 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470545:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784675.470549:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470555:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115800 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:2.0:1787784675.470750:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470759:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787784675.470766:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784675.470778:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787784675.470792:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470798:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787784675.470813:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470820:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98952a115800/0xb0ce63137acc1c5b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc1c62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.470833:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784675.470834:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784675.470836:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784675.470843:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784675.470854:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668342528:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787784675.470882:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784675.470975:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668342528 00000100:00100000:2.0:1787784675.470980:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668342528 00010000:00010000:2.0:1787784675.470996:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989539135600/0xb0ce63137acc1c62 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c5b expref: 18 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.471015:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1787784675.471021:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+18:3279:x1874627668342528:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787784675.471029:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784675.471034:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989539135600/0xb0ce63137acc1c62 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c5b expref: 18 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.471043:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989539135600/0xb0ce63137acc1c62 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc1c5b expref: 18 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.471065:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784675.471095:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+18:3279:x1874627668342528:12345-0@lo:103:ldlm_bl.0 Request processed in 73us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784675.471102:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1787784675.471131:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668342528:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784675.471264:0:17448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989500030600/0xb0ce63137acc1c9a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc1c93 expref: 17 pid: 5805 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.471275:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6ab840 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668340864:0@lo:104: 00010000:00010000:3.0:1787784675.471285:0:17448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989500031a00/0xb0ce63137acc1c8c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc1c85 expref: 16 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.471293:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028da40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668341376:0@lo:104: 00000100:00100000:3.0:1787784675.471301:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668341504:0@lo:104: 00000100:00100000:3.0:1787784675.471309:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ad2c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668341888:0@lo:104: 00010000:00010000:3.0:1787784675.471317:0:17448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989539135600/0xb0ce63137acc1c62 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc1c5b expref: 15 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784675.471325:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ade00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668342016:0@lo:104: 00010000:00010000:0.0:1787784675.472030:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989539135e00/0xb0ce63137acc1c7e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc1c77 expref: 14 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.472047:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989539135e00/0xb0ce63137acc1c7e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc1c77 expref: 14 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.472058:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784675.472094:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ed850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+13:3277:x1874627668341760:12345-0@lo:103:ldlm_bl.0 Request processed in 6172us (6279us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784675.472102:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00010000:00010000:1.0:1787784675.472199:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784675.472418:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668342400:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784675.472447:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784675.472492:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668341760:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787784675.472737:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668342400 00000100:00100000:2.0:1787784675.472744:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668342400 00010000:00010000:2.0:1787784675.472770:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989500031000/0xb0ce63137acc1c70 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c69 expref: 13 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784675.472801:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1787784675.472809:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+13:3277:x1874627668342400:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787784675.472819:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784675.472824:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989500031000/0xb0ce63137acc1c70 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc1c69 expref: 13 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.472834:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989500031000/0xb0ce63137acc1c70 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc1c69 expref: 13 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.472844:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989500031000/0xb0ce63137acc1c70 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc1c69 expref: 13 pid: 5803 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784675.472855:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784675.473034:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+12:3277:x1874627668342400:12345-0@lo:103:ldlm_bl.0 Request processed in 224us (585us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784675.473041:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1787784675.473829:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668342400:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784675.474790:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784675.474879:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668342144:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784675.475117:0:17448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17448:1874627668342272:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784675.475222:0:17448:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784675.475625:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002688 00000100:00100000:0.0:1787784675.475633:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002688 00010000:00010000:0.0:1787784675.475667:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff98953efb2000/0xb0ce63137acc1c54 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba7 expref: 12 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784675.475733:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:0.0:1787784675.475742:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:2357:x1874627658002688:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784675.475756:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784675.475762:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff98953efb2000/0xb0ce63137acc1c54 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba7 expref: 12 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.475774:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff98953efb2000/0xb0ce63137acc1c54 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba7 expref: 12 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.475791:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff98953efb2000/0xb0ce63137acc1c54 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba7 expref: 12 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.475804:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784675.475858:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:2357:x1874627658002688:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 117us (1073us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784675.475867:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:1.0:1787784675.478777:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784675.478883:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002816 00000100:00100000:0.0:1787784675.478892:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002816 00010000:00010000:0.0:1787784675.478997:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff98953efb3400/0xb0ce63137acc1c4d lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba0 expref: 11 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784675.479035:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 211 00000100:00100000:0.0:1787784675.479044:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:2356:x1874627658002816:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784675.479680:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784675.479688:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff98953efb3400/0xb0ce63137acc1c4d lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba0 expref: 11 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479702:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff98953efb3400/0xb0ce63137acc1c4d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba0 expref: 11 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479720:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff98953efb3400/0xb0ce63137acc1c4d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afba0 expref: 11 pid: 5805 timeout: 566 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479736:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479743:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479759:0:11303:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479765:0:11303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479772:0:11303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479778:0:11303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479784:0:11303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eaa00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784675.479787:0:11303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479805:0:11303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479817:0:11303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784675.479828:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784675.479885:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:2356:x1874627658002816:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 842us (1116us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784675.479893:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 211 00010000:00010000:1.0:1787784675.480955:0:17448:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784675.480969:0:17448:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784675.480975:0:17448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4b32 lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784676.273395:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784676.849476:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668342656:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784676.849615:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784676.849894:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668342656 00000100:00100000:2.0:1787784676.849911:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668342656 00000100:00100000:1.0:1787784676.849945:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668342912:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.849986:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784676.849988:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:2.0:1787784676.849999:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668342656:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784676.850011:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1787784676.850019:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a91a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668342784:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.850051:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1787784676.850813:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784676.850883:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668342656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 884us (1268us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784676.850891:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00080000:1.0:1787784676.865399:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784676.865435:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668342912 00000100:00100000:1.0:1787784676.866082:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668342912 00000100:00100000:1.0:1787784676.866407:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1787784676.866422:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668342912:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.866589:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668342912:12345-0@lo:400:kworker.0 Request processed in 170us (16601us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784676.866597:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1787784676.866633:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668342784 00000100:00100000:1.0:1787784676.866638:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668342784 00000100:00100000:1.0:1787784676.866667:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:1.0:1787784676.866672:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+12:3277:x1874627668342784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.866704:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+12:3277:x1874627668342784:12345-0@lo:400:kworker.0 Request processed in 31us (16652us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784676.866711:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:1.0:1787784676.866806:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a91a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668342784:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.866822:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668343296:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.866862:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784676.866903:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668342912:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.867094:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668343168:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.867127:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784676.867181:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967335 gen 2 00000004:00080000:1.0:1787784676.867187:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967335 committed transno 4294967335 00000100:00100000:1.0:1787784676.867215:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668342656:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784676.868083:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343296 00000100:00100000:3.0:1787784676.868094:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343296 00000100:00100000:3.0:1787784676.868175:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1787784676.868185:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668343296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784676.868267:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668343296:12345-0@lo:400:kworker.0 Request processed in 84us (1404us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784676.868274:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1787784676.868300:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343168 00000100:00100000:3.0:1787784676.868304:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343168 00000100:00100000:3.0:1787784676.868333:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:3.0:1787784676.868339:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668343168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784676.868603:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668343168:12345-0@lo:400:kworker.0 Request processed in 263us (1474us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784676.868610:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00080000:1.0:1787784676.868703:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784676.868710:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784676.868744:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668343296:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.868784:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668343168:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.868796:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784676.868807:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668343040:0@lo:400:kworker.0 00000100:00100000:1.0:1787784676.868842:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784676.868872:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343040 00000100:00100000:1.0:1787784676.868875:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343040 00000100:00100000:1.0:1787784676.868908:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:1.0:1787784676.868915:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668343040:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1787784676.875790:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784676.875891:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668343040:12345-0@lo:400:kworker.0 Request processed in 6967us (7040us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784676.875906:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:1.0:1787784676.876012:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668343040:0@lo:400:kworker.0 00000020:00080000:2.0:1787784676.903327:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 10 batchid updated 00000001:00080000:2.0:1787784676.903385:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967774 is committed 00000020:00080000:3.0:1787784676.903476:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 10 00000020:00080000:3.0:1787784676.903489:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 10 10000000:01000000:2.0:1787784677.041800:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787784677.041810:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989518f73800 from 42 10000000:01000000:2.0:1787784677.041816:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787784677.041880:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.041895:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.041909:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784677.041939:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668343424:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787784677.042904:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784677.042989:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502f78a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.043014:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343424 00000100:00100000:1.0:1787784677.043025:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343424 00000100:00100000:1.0:1787784677.043100:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:1.0:1787784677.043111:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+12:5808:x1874627668343424:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784677.043327:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784677.043360:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4b39 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.043394:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 13 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.043407:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 13 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.043417:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad800, rc 0) 00000100:00100000:1.0:1787784677.043489:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668343424:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 380us (590us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.043496:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:2.0:1787784677.043631:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668343424:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784677.043649:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.043662:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784677.043744:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668343552:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784677.043997:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784677.044043:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502f78a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.044103:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343552 00000100:00100000:1.0:1787784677.044108:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343552 00000100:00100000:1.0:1787784677.044141:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:1.0:1787784677.044170:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668343552:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.044637:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668343552:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 467us (637us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.044645:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1787784677.047126:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668343552:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.047228:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668343680:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.047326:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.047432:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32840 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.047662:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343680 00000100:00100000:1.0:1787784677.047670:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343680 00000100:00100000:1.0:1787784677.047862:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:1.0:1787784677.047874:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668343680:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.048503:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668343680:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 629us (1174us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.048524:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:3.0:1787784677.049026:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668343680:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.053582:0:17450:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17450:1874627668343808:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.053697:0:17450:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.053985:0:17450:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784677.054073:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343808 00000100:00100000:2.0:1787784677.054082:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343808 00000100:00100000:2.0:1787784677.054317:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:2.0:1787784677.054330:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:17450:x1874627668343808:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.055883:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:17450:x1874627668343808:12345-0@lo:502:llog_process_th.0 Request processed in 1400us (2031us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784677.055970:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:1.0:1787784677.056552:0:17450:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17450:1874627668343808:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784677.057667:0:17450:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17450:1874627668343936:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784677.057786:0:17450:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784677.057940:0:17450:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784677.058514:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668343936 00000100:00100000:2.0:1787784677.058527:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668343936 00000100:00100000:2.0:1787784677.058655:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:2.0:1787784677.058668:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:17450:x1874627668343936:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784677.059085:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:17450:x1874627668343936:12345-0@lo:502:llog_process_th.0 Request processed in 419us (1299us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.059095:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:3.0:1787784677.059257:0:17450:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17450:1874627668343936:0@lo:502:llog_process_th.0 00000040:00080000:3.0:1787784677.059305:0:17450:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 45 count 45 00000020:01000000:0.0:1787784677.061320:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 42-44 (rc=0) 10000000:01000000:0.0:1787784677.061383:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787784677.061400:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.061414:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784677.061428:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:0.0:1787784677.061431:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:0.0:1787784677.061434:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:0.0:1787784677.061489:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.061498:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.061510:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784677.061565:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344064:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.061652:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.068243:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.068282:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344064 00000100:00100000:1.0:1787784677.068295:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344064 00000100:00100000:1.0:1787784677.068382:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:1.0:1787784677.068394:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668344064:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784677.068417:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784677.068517:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4b47 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.068557:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 14 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.068572:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 14 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.068583:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac600, rc 0) 00000100:00100000:1.0:1787784677.068674:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668344064:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 282us (7022us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.068683:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:0.0:1787784677.069030:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344064:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784677.069052:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.069063:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784677.069366:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344192:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.069413:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784677.069869:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344192 00000100:00100000:1.0:1787784677.069876:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344192 00000100:00100000:1.0:1787784677.069919:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:1.0:1787784677.069929:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668344192:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.070298:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668344192:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 368us (883us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.070306:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:0.0:1787784677.071133:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.071559:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344192:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.071612:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344320:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.071671:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.071698:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784677.074591:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344320 00000100:00100000:2.0:1787784677.074612:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344320 00000100:00100000:2.0:1787784677.074706:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1787784677.074952:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668344320:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784677.075536:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668344320:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 587us (3861us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.075546:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:0.0:1787784677.075697:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344320:0@lo:503:ll_cfg_requeue.0 00000040:00080000:1.0:1787784677.077010:0:17451:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:0.0:1787784677.077758:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:0.0:1787784677.077799:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:0.0:1787784677.077814:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.077828:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784677.077857:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787784677.077860:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989538a62800 from 42 10000000:01000000:0.0:1787784677.077864:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787784677.077911:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.077920:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.077933:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784677.077962:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344448:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.078053:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.078105:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784677.078339:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344448 00000100:00100000:2.0:1787784677.078359:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344448 00000100:00100000:2.0:1787784677.078419:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:2.0:1787784677.078429:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668344448:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784677.078457:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784677.078473:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4b55 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.078517:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.078529:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.078538:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb600, rc 0) 00000100:00100000:2.0:1787784677.078592:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668344448:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 164us (540us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.078600:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1787784677.078994:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344448:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784677.079012:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.079082:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784677.079339:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344576:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.079392:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.079429:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.079555:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344576 00000100:00100000:1.0:1787784677.079561:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344576 00000100:00100000:1.0:1787784677.079607:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1787784677.079618:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668344576:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.087744:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668344576:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 8101us (8324us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.087766:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:0.0:1787784677.092494:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344576:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.092585:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344704:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.092701:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.092763:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.092902:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344704 00000100:00100000:1.0:1787784677.092912:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344704 00000100:00100000:1.0:1787784677.092983:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1787784677.092994:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668344704:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.093304:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668344704:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 309us (602us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.093313:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:0.0:1787784677.094489:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668344704:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.096579:0:17452:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17452:1874627668344832:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.096681:0:17452:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.096735:0:17452:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.096999:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344832 00000100:00100000:0.0:1787784677.097007:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344832 00000100:00100000:0.0:1787784677.097050:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1787784677.097060:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17452:x1874627668344832:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784677.097624:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17452:x1874627668344832:12345-0@lo:502:llog_process_th.0 Request processed in 562us (940us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.097638:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:3.0:1787784677.098082:0:17452:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17452:1874627668344832:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.098132:0:17452:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17452:1874627668344960:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.098196:0:17452:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.098222:0:17452:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.098534:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668344960 00000100:00100000:0.0:1787784677.098540:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668344960 00000100:00100000:0.0:1787784677.098576:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:0.0:1787784677.098584:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:17452:x1874627668344960:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784677.098888:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:17452:x1874627668344960:12345-0@lo:502:llog_process_th.0 Request processed in 304us (691us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.098895:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:3.0:1787784677.099055:0:17452:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17452:1874627668344960:0@lo:502:llog_process_th.0 00000040:00080000:3.0:1787784677.099077:0:17452:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 45 count 45 00000020:01000000:0.0:1787784677.099240:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 42-44 (rc=0) 10000000:01000000:0.0:1787784677.099260:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787784677.099268:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.099279:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784677.099289:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:0.0:1787784677.099292:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:0.0:1787784677.099302:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:0.0:1787784677.099328:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.099337:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.099410:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784677.099430:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345088:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.099535:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.099603:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e780 going to sleep for 11 seconds 00000100:00100000:3.0:1787784677.101736:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345088 00000100:00100000:3.0:1787784677.101748:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345088 00000100:00100000:3.0:1787784677.101864:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1787784677.101875:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668345088:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784677.101891:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784677.101909:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4b63 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.101961:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 16 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.101972:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 16 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.101982:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2400, rc 0) 00000100:00100000:3.0:1787784677.102065:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668345088:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 192us (2536us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784677.102073:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:2.0:1787784677.102707:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345088:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784677.102747:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.102761:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784677.102908:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345216:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.103057:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.103134:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.103293:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345216 00000100:00100000:0.0:1787784677.103300:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345216 00000100:00100000:0.0:1787784677.103430:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:0.0:1787784677.103439:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668345216:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.103858:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668345216:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 419us (800us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.103866:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1787784677.103916:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345216:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.104004:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345344:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.104095:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.104127:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345344 00000100:00100000:0.0:1787784677.104134:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345344 00000100:00100000:0.0:1787784677.104212:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:0.0:1787784677.104223:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668345344:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.104225:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.104836:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668345344:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 615us (740us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.104844:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:1.0:1787784677.104892:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345344:0@lo:503:ll_cfg_requeue.0 00000040:00080000:1.0:1787784677.107293:0:17453:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000020:01000000:3.0:1787784677.108886:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:3.0:1787784677.108940:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:3.0:1787784677.108954:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.108968:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784677.108979:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787784677.108983:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9895054dd800 from 42 10000000:01000000:3.0:1787784677.108985:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787784677.109023:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.109030:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.109055:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784677.109082:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345472:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.109174:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.109224:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.109270:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345472 00000100:00100000:0.0:1787784677.109302:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345472 00000100:00100000:0.0:1787784677.109381:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:0.0:1787784677.109393:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668345472:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784677.109412:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784677.109432:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4b71 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.109466:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.109477:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.109488:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbdc00, rc 0) 00000100:00100000:0.0:1787784677.109629:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668345472:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 237us (481us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.109638:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:3.0:1787784677.112452:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345472:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784677.112493:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.112510:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784677.112621:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345600:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.112694:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.112829:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.117984:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345600 00000100:00100000:0.0:1787784677.118005:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345600 00000100:00100000:0.0:1787784677.118126:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:0.0:1787784677.118140:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668345600:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.120172:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668345600:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 2001us (7444us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.120187:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:1.0:1787784677.123081:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345600:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.123319:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345728:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.123428:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784677.123542:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd780 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.130180:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784677.130300:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345728 00000100:00100000:2.0:1787784677.130314:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345728 00000100:00100000:2.0:1787784677.130379:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658002944 00000100:00100000:2.0:1787784677.130382:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658002944 00000100:00100000:2.0:1787784677.130403:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1787784677.130414:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668345728:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.130990:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:0.0:1787784677.131005:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:5597:x1874627658002944:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784677.131017:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784677.131050:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b094c expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.131082:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 11 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.131095:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 11 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.131105:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc600, rc 0) 00000100:00100000:0.0:1787784677.135169:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658002944:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 4129us (4966us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.135185:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:2.0:1787784677.139291:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668345728:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 8866us (15850us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.139309:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:0.0:1787784677.141640:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784677.141916:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003072 00000100:00100000:2.0:1787784677.141940:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003072 00000100:00100000:2.0:1787784677.142082:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 375 00000100:00100000:2.0:1787784677.142095:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658003072:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784677.143428:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658003072:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 1330us (1797us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.143441:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 375 00000100:00100000:3.0:1787784677.146717:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668345728:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.148782:0:17454:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17454:1874627668345856:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.149067:0:17454:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.149132:0:17454:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.149228:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345856 00000100:00100000:0.0:1787784677.149237:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345856 00000100:00100000:0.0:1787784677.149303:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:0.0:1787784677.149314:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17454:x1874627668345856:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784677.149649:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17454:x1874627668345856:12345-0@lo:502:llog_process_th.0 Request processed in 334us (581us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.149658:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:3.0:1787784677.151553:0:17454:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17454:1874627668345856:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.151652:0:17454:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17454:1874627668345984:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784677.151729:0:17454:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.151841:0:17454:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.152503:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668345984 00000100:00100000:0.0:1787784677.152512:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668345984 00000100:00100000:0.0:1787784677.152583:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:0.0:1787784677.152594:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17454:x1874627668345984:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784677.153275:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17454:x1874627668345984:12345-0@lo:502:llog_process_th.0 Request processed in 680us (1544us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.153286:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1787784677.154092:0:17454:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17454:1874627668345984:0@lo:502:llog_process_th.0 00000040:00080000:2.0:1787784677.154138:0:17454:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 45 count 45 00000020:01000000:0.0:1787784677.154534:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 42-44 (rc=0) 10000000:01000000:0.0:1787784677.154563:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787784677.154575:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.154588:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784677.154599:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:0.0:1787784677.154603:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 37 10000000:01000000:0.0:1787784677.154606:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:0.0:1787784677.154644:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.154652:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.154663:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784677.154685:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346112:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.154737:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.155487:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784677.155671:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668346112 00000100:00100000:2.0:1787784677.155680:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668346112 00000100:00100000:2.0:1787784677.155780:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 378 00000100:00100000:2.0:1787784677.155792:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668346112:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784677.155809:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784677.155827:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4b86 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.155859:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.155873:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.158003:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb200, rc 0) 00000100:00100000:2.0:1787784677.158114:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668346112:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 2320us (3372us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.158124:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 378 00000100:00100000:0.0:1787784677.158277:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346112:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784677.158305:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.158320:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784677.158445:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346240:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.158482:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784677.158619:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784677.164611:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce780 going to sleep for 11 seconds 00000100:00100000:3.0:1787784677.167038:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668346240 00000100:00100000:3.0:1787784677.167061:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668346240 00000100:00100000:3.0:1787784677.167161:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003200 00000100:00100000:3.0:1787784677.167166:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003200 00000100:00100000:3.0:1787784677.167194:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:3.0:1787784677.167205:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668346240:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.168455:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668346240:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 1244us (9964us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784677.168469:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00100000:1.0:1787784677.168495:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346240:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.168525:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 380 00000100:00100000:3.0:1787784677.168538:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658003200:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.168569:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346368:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.168627:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784677.168657:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784677.168697:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668346368 00000100:00100000:2.0:1787784677.168711:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668346368 00000100:00100000:2.0:1787784677.168746:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1787784677.168881:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658003200:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 345us (10262us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784677.168888:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 380 00000100:00100000:2.0:1787784677.169694:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668346368:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784677.170224:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668346368:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 539us (1595us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.170234:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:1.0:1787784677.172585:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346368:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.175677:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784677.175787:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003328 00000100:00100000:3.0:1787784677.175805:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003328 00000100:00100000:3.0:1787784677.175963:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 382 00000100:00100000:3.0:1787784677.175976:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22466:x1874627658003328:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:3.0:1787784677.176437:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22466:x1874627658003328:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 463us (773us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784677.176447:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 382 00000100:00100000:1.0:1787784677.179660:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784677.179707:0:17455:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ade00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17455:1874627668346496:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784677.179751:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003456 00000100:00100000:2.0:1787784677.179759:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003456 00000100:00100000:3.0:1787784677.179809:0:17455:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.180007:0:17455:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32240 going to sleep for 11 seconds 00000100:00100000:2.0:1787784677.180106:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668346496 00000100:00100000:2.0:1787784677.180110:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668346496 00000100:00100000:2.0:1787784677.180133:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 383 00000100:00100000:2.0:1787784677.180143:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22466:x1874627658003456:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:1.0:1787784677.180478:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1787784677.180545:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:17455:x1874627668346496:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784677.180548:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22466:x1874627658003456:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 404us (898us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.180560:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 383 00000100:00100000:2.0:1787784677.180877:0:17455:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ade00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17455:1874627668346496:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784677.180889:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:17455:x1874627668346496:12345-0@lo:502:llog_process_th.0 Request processed in 295us (1025us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.180897:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000020:01000000:2.0:1787784677.180931:0:17455:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787784677.180946:0:17455:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787784677.180949:0:17455:0:(obd_config.c:1563:class_process_config()) marker 29 (0x1) lustre-MDT0000-m new lustre.test_11c 00000020:00000080:2.0:1787784677.180957:0:17455:0:(obd_config.c:1472:class_process_config()) processing cmd: ce020 00020000:01000000:2.0:1787784677.181044:0:17455:0:(lod_pool.c:475:lod_pool_new()) test_11c is pool #1 00080000:10000000:3.0:1787784677.185027:0:17455:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x37:0x0] (8/8/4) registered 00000100:00100000:2.0:1787784677.194028:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784677.196625:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003584 00000100:00100000:0.0:1787784677.196644:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003584 00000100:00100000:0.0:1787784677.196741:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 385 00000100:00100000:0.0:1787784677.196753:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658003584:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784677.196776:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784677.196808:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0953 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.196844:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.196856:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.196866:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784677.196953:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658003584:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 202us (2939us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.196961:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 385 00000100:00100000:3.0:1787784677.203823:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784677.203978:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003712 00000100:00100000:0.0:1787784677.203993:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003712 00000100:00100000:0.0:1787784677.204059:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:0.0:1787784677.204070:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658003712:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784677.204465:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658003712:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 396us (656us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.204483:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:0.0:1787784677.224641:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784677.226293:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003840 00000100:00100000:2.0:1787784677.226311:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003840 00000100:00100000:2.0:1787784677.226395:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 387 00000100:00100000:2.0:1787784677.226408:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658003840:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784677.226692:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658003840:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 285us (2063us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784677.226710:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 387 00000100:00100000:2.0:1787784677.235292:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784677.236967:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658003968 00000100:00100000:3.0:1787784677.236986:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658003968 00000100:00100000:3.0:1787784677.237057:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 388 00000100:00100000:3.0:1787784677.237069:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22467:x1874627658003968:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:3.0:1787784677.237475:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22467:x1874627658003968:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 407us (2196us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784677.237485:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 388 00000100:00100000:1.0:1787784677.241624:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784677.241835:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004096 00000100:00100000:3.0:1787784677.241848:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004096 00000100:00100000:3.0:1787784677.241953:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 389 00000100:00100000:3.0:1787784677.241965:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22467:x1874627658004096:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:3.0:1787784677.242370:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22467:x1874627658004096:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 406us (759us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784677.242380:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 389 00000100:00100000:3.0:1787784677.252650:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784677.256043:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004224 00000100:00100000:1.0:1787784677.256064:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004224 00000100:00100000:1.0:1787784677.256144:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 390 00000100:00100000:1.0:1787784677.256184:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658004224:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784677.256207:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784677.256232:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b095a expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.256280:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.256294:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784677.256305:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac200, rc 0) 00000100:00100000:1.0:1787784677.256833:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658004224:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 647us (4197us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.256842:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 390 00000100:00100000:0.0:1787784677.262108:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784677.262397:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004352 00000100:00100000:3.0:1787784677.262410:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004352 00000100:00100000:3.0:1787784677.262481:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:3.0:1787784677.262493:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658004352:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.262871:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658004352:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 378us (773us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784677.262881:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 391 00080000:10000000:2.0:1787784677.271523:0:17455:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x38:0x0] (8/8/4) registered 00010000:00010000:3.0:1787784677.300112:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00080000:10000000:2.0:1787784677.312129:0:17455:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x39:0x0] (8/8/4) registered 00000100:00100000:2.0:1787784677.354477:0:17455:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17455:1874627668346624:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784677.362547:0:17455:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784677.362609:0:17455:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.362704:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668346624 00000100:00100000:1.0:1787784677.362716:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668346624 00000100:00100000:1.0:1787784677.362922:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:1.0:1787784677.363451:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:17455:x1874627668346624:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784677.364250:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:17455:x1874627668346624:12345-0@lo:502:llog_process_th.0 Request processed in 803us (1707us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784677.364261:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:0.0:1787784677.365032:0:17455:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17455:1874627668346624:0@lo:502:llog_process_th.0 00000020:01000000:0.0:1787784677.365081:0:17455:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:0.0:1787784677.365090:0:17455:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:0.0:1787784677.365093:0:17455:0:(obd_config.c:1563:class_process_config()) marker 29 (0x2) lustre-MDT0000-m new lustre.test_11c 00000040:00080000:0.0:1787784677.365099:0:17455:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 40 count 40 00000020:01000000:2.0:1787784677.365907:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 37-39 (rc=0) 10000000:01000000:2.0:1787784677.366022:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:2.0:1787784677.366269:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.366287:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784677.366302:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787784677.366306:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff989508964d28 from 1 10000000:01000000:2.0:1787784677.366310:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1787784677.366617:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.366629:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.366643:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784677.366675:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346752:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787784677.367014:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784677.367096:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e840 going to sleep for 11 seconds 00000100:00100000:0.0:1787784677.367471:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668346752 00000100:00100000:0.0:1787784677.367479:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668346752 00000100:00100000:0.0:1787784677.367535:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:0.0:1787784677.367545:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668346752:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784677.367562:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784677.367580:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4ba2 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.367612:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 19 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.367624:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 19 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784677.367636:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbde00, rc 0) 00000100:00100000:0.0:1787784677.367892:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668346752:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 347us (878us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784677.367900:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1787784677.372127:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346752:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784677.372239:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784677.372255:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784677.373108:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ad2c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346880:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784677.373257:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784677.373395:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784677.377040:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668346880 00000100:00100000:1.0:1787784677.377061:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668346880 00000100:00100000:1.0:1787784677.377230:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1787784677.377243:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668346880:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784677.377743:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668346880:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 501us (4485us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784677.377762:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:3.0:1787784677.378092:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ad2c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668346880:0@lo:501:ll_cfg_requeue.0 10000000:01000000:3.0:1787784677.378169:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:3.0:1787784677.378183:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784677.378195:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.324754:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1787784678.631472:0:17504:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007ffc247f6bc0 20000000:00000080:1.0:1787784678.631493:0:17504:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff98953f00b000 uarg=0000000000000000 00000040:00100000:1.0:1787784678.632285:0:17504:0:(llog.c:1072:llog_reverse_process()) index 44 : idx 1 20000000:01000000:1.0:1787784678.632294:0:17504:0:(mgs_llog.c:963:mgs_search_pool_cb()) Matched pool rec 43 cmd:0xce020 lustre.test_11c 00000040:00080000:1.0:1787784678.645472:0:17504:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].45, 224 off16008 20000000:01000000:1.0:1787784678.645513:0:17504:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce021 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1787784678.645609:0:17504:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].46, 136 off16144 00000040:00080000:1.0:1787784678.647480:0:17504:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].48, 224 off16608 00000040:00080000:1.0:1787784678.652012:0:17504:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].40, 224 off15280 20000000:01000000:1.0:1787784678.652043:0:17504:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce021 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1787784678.652124:0:17504:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].41, 144 off15424 00000040:00080000:1.0:1787784678.652215:0:17504:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].42, 224 off15648 10000000:01000000:1.0:1787784678.652240:0:17504:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:1.0:1787784678.652259:0:17504:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.652278:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.652286:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.652295:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.652303:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.652309:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655872:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655910:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655918:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655924:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655931:0:17504:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655939:0:17504:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655982:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.655998:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 18 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.656012:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347008:0@lo:104: 00000100:00100000:1.0:1787784678.656076:0:17504:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787784678.656161:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656172:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 12 pid: 5805 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.656194:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347136:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784678.656252:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656261:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 12 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.656269:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347264:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784678.656311:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656319:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 18 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.656326:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347392:0@lo:104: 00000100:00100000:1.0:1787784678.656361:0:17504:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787784678.656393:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656401:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 12 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.656418:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347520:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784678.656448:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656456:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 18 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.656472:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347648:0@lo:104: 00000100:00100000:1.0:1787784678.656496:0:17504:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784678.656537:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668347008 00000100:00100000:1.0:1787784678.656544:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668347008 00000100:00100000:1.0:1787784678.656583:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668347392 00000100:00100000:1.0:1787784678.656586:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668347392 00000100:00100000:1.0:1787784678.656595:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668347648 00000100:00100000:1.0:1787784678.656597:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668347648 00000100:00100000:1.0:1787784678.656609:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 35 00000100:00100000:1.0:1787784678.656618:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+5:17504:x1874627668347008:12345-0@lo:104: 00010000:00010000:1.0:1787784678.656630:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656673:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787784678.656678:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+5:17504:x1874627668347008:12345-0@lo:104: Request processed in 61us (602us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784678.656685:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 35 00000100:00100000:1.0:1787784678.656703:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 36 00000100:00100000:1.0:1787784678.656708:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17504:x1874627668347392:12345-0@lo:104: 00010000:00010000:1.0:1787784678.656715:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656746:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:1.0:1787784678.656751:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17504:x1874627668347392:12345-0@lo:104: Request processed in 43us (390us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784678.656757:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 36 00000100:00100000:1.0:1787784678.656772:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:1.0:1787784678.656785:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17504:x1874627668347648:12345-0@lo:104: 00010000:00010000:1.0:1787784678.656795:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656824:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:1.0:1787784678.656828:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17504:x1874627668347648:12345-0@lo:104: Request processed in 45us (332us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784678.656833:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00010000:00010000:1.0:1787784678.656852:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787784678.656857:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656864:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:1.0:1787784678.656868:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656878:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787784678.656886:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784678.656892:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787784678.656917:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656924:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787784678.656942:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784678.656946:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:1.0:1787784678.656959:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4ba2 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4ba9 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784678.656959:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668347776:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784678.656971:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787784678.656975:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.656980:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:1.0:1787784678.656983:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784678.656989:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787784678.656990:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787784678.656996:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784678.657000:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787784678.657014:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.657019:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787784678.657023:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668347776 00000100:00100000:0.0:1787784678.657026:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668347776 00010000:00010000:1.0:1787784678.657031:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.657038:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4b86 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b8d expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784678.657043:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 19 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.657046:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784678.657054:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.657060:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc1da1c20) 00000100:00100000:0.0:1787784678.657062:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 10000000:00010000:1.0:1787784678.657063:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784678.657068:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668347776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784678.657070:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787784678.657076:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784678.657078:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 10000000:01000000:1.0:1787784678.657081:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784678.657082:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 19 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784678.657091:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 19 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.657092:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.657098:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787784678.657104:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784678.657108:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784678.657109:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98953f5b3400/0xb0ce63137acc4b71 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b78 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.657118:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784678.657120:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784678.657122:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784678.657136:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668348032:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784678.657165:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668347776:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (143us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784678.657172:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1787784678.657185:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784678.658755:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348032 00000100:00100000:0.0:1787784678.658769:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348032 00010000:00010000:0.0:1787784678.658809:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 6/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 19 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784678.658838:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:0.0:1787784678.658846:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057edc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668348032:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787784678.658858:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784678.658874:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 6/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 19 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784678.658883:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 19 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784678.658902:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784678.658946:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057edc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668348032:12345-0@lo:103:ldlm_bl.0 Request processed in 102us (1760us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784678.658953:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00010000:00010000:3.0:1787784678.658970:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784678.658984:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668347904:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.659014:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784678.659029:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668347776:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.659061:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668347904 00000100:00100000:3.0:1787784678.659064:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668347904 00010000:00010000:3.0:1787784678.659078:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 19 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.659103:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1787784678.659109:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668347904:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784678.659118:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784678.659122:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 19 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.659130:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 19 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.659140:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784678.659199:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668347904:12345-0@lo:103:ldlm_bl.0 Request processed in 90us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784678.659207:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1787784678.659231:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668347904:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784678.663180:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668348032:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784678.663288:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.663302:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 18 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.663326:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348160:0@lo:104: 00000100:00100000:1.0:1787784678.663393:0:17504:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784678.663704:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.663714:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 18 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.663722:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021afc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348288:0@lo:104: 00000100:00100000:3.0:1787784678.663748:0:17504:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784678.673509:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.673528:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 18 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.673543:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acb40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348416:0@lo:104: 00000100:00100000:3.0:1787784678.673598:0:17504:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784678.673774:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784678.673798:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784678.673822:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784678.673845:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004608 00000100:00100000:1.0:1787784678.673852:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004608 00010000:00010000:1.0:1787784678.673883:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 13 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.673909:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004736 00000100:00100000:1.0:1787784678.673912:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004736 00010000:00010000:1.0:1787784678.673923:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 14 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.673932:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004480 00000100:00100000:1.0:1787784678.673934:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004480 00010000:00010000:1.0:1787784678.673944:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 15 pid: 5805 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.673956:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 215 00000100:00100000:1.0:1787784678.673964:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+15:2357:x1874627658004608:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784678.673975:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784678.673995:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 15 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.674005:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 15 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.674024:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784678.674065:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+15:2357:x1874627658004608:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 102us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784678.674073:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 215 00000100:00100000:1.0:1787784678.674092:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 216 00000100:00100000:1.0:1787784678.674098:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:2357:x1874627658004736:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784678.674107:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784678.674112:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 14 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.674120:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 14 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.674129:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784678.674165:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:2357:x1874627658004736:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 48us (348us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784678.674171:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 216 00000100:00100000:1.0:1787784678.674184:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 217 00000100:00100000:1.0:1787784678.674191:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658004480:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784678.674198:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784678.674202:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 13 pid: 5805 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.674216:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 13 pid: 5805 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784678.674225:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784678.674243:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658004480:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 53us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784678.674250:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 217 00000100:00100000:2.0:1787784678.682296:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348160 00000100:00100000:2.0:1787784678.682310:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348160 00010000:00010000:3.0:1787784678.684125:0:17504:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.685890:0:17504:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 18 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.685911:0:17504:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ac780 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348544:0@lo:104: 00000100:00100000:3.0:1787784678.685985:0:17504:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784678.689910:0:17504:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd4a80 going to sleep for 11 seconds 00010000:00010000:3.0:1787784678.689968:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4ba9 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4ba2 expref: 18 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.689987:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347008:0@lo:104: 00010000:00010000:3.0:1787784678.690003:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894010ac200/0xb0ce63137acc4b9b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b095a expref: 12 pid: 5805 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.690011:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347136:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784678.690021:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4b94 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0953 expref: 11 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.690028:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347264:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784678.690037:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4b8d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4b86 expref: 17 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.690045:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347392:0@lo:104: 00010000:00010000:3.0:1787784678.690054:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4b7f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b094c expref: 10 pid: 5803 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.690062:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347520:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784678.690072:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4b78 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4b71 expref: 16 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.690079:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668347648:0@lo:104: 00000100:00100000:0.0:1787784678.691664:0:5794:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348288 00000100:00100000:3.0:1787784678.691862:0:8493:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787784678.691884:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348416 00000100:00100000:3.0:1787784678.691892:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348416 00000100:00100000:3.0:1787784678.691937:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348544 00000100:00100000:3.0:1787784678.691952:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348544 00000100:00100000:3.0:1787784678.691972:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:3.0:1787784678.691981:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+5:17504:x1874627668348160:12345-0@lo:104: 00010000:00010000:3.0:1787784678.691998:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692089:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787784678.692095:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+5:17504:x1874627668348160:12345-0@lo:104: Request processed in 116us (28702us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784678.692103:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000100:00100000:3.0:1787784678.692125:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:3.0:1787784678.692131:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17504:x1874627668348416:12345-0@lo:104: 00010000:00010000:3.0:1787784678.692139:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692194:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:3.0:1787784678.692199:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17504:x1874627668348416:12345-0@lo:104: Request processed in 69us (18602us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784678.692206:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000100:00100000:3.0:1787784678.692220:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:3.0:1787784678.692226:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401d16050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17504:x1874627668348544:12345-0@lo:104: 00010000:00010000:3.0:1787784678.692234:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692261:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:3.0:1787784678.692266:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401d16050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17504:x1874627668348544:12345-0@lo:104: Request processed in 41us (6282us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784678.692271:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00010000:00010000:3.0:1787784678.692295:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787784678.692301:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692309:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbca00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784678.692313:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692321:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784678.692342:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784678.692349:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784678.692386:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692393:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787784678.692416:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692423:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4b63 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b6a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692437:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787784678.692440:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692444:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784678.692447:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692460:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784678.692467:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784678.692474:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784678.692490:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692496:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787784678.692502:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692509:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbd600/0xb0ce63137acc4b47 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b4e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692519:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784678.692521:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692527:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea600 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784678.692530:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692537:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784678.692542:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784678.692553:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784678.692568:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692573:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787784678.692587:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692593:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff9894032ea600/0xb0ce63137acc4b39 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b40 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692601:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784678.692602:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784678.692604:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784678.692624:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668348672:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.692662:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784678.692712:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784678.692718:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668348800:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.692741:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784678.692768:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668348928:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.692793:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784678.692827:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348672 00000100:00100000:3.0:1787784678.692831:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348672 00010000:00010000:3.0:1787784678.692850:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 16 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.692869:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348800 00000100:00100000:3.0:1787784678.692872:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348800 00010000:00010000:3.0:1787784678.692891:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 17 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.692902:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668348928 00000100:00100000:3.0:1787784678.692904:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348928 00010000:00010000:3.0:1787784678.692913:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 18 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784678.692922:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1787784678.692928:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+18:3276:x1874627668348672:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784678.692937:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784678.692942:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 18 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692952:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 18 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.692968:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784678.692994:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+18:3276:x1874627668348672:12345-0@lo:103:ldlm_bl.0 Request processed in 66us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784678.693001:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1787784678.693015:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1787784678.693020:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+17:3276:x1874627668348800:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784678.693028:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784678.693031:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 17 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.693039:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 17 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.693059:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784678.693081:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+17:3276:x1874627668348800:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784678.693087:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1787784678.693099:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1787784678.693105:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ecc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+16:3278:x1874627668348928:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784678.693112:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784678.693116:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 16 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.693123:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 16 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784678.693132:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784678.693183:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ecc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+16:3278:x1874627668348928:12345-0@lo:103:ldlm_bl.0 Request processed in 79us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784678.693189:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1787784678.693239:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668348800:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.693247:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668348672:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.693270:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668348928:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787784678.693553:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff98953f5b2400/0xb0ce63137acc4b6a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4b63 expref: 15 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784678.693566:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348160:0@lo:104: 00010000:00010000:2.0:1787784678.693575:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894010ac600/0xb0ce63137acc4b4e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4b47 expref: 14 pid: 5804 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784678.693583:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acb40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348416:0@lo:104: 00010000:00010000:2.0:1787784678.693593:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894010ad800/0xb0ce63137acc4b40 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4b39 expref: 13 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784678.693599:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ac780 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348544:0@lo:104: 00000100:00100000:0.0:1787784678.693652:0:5794:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668348288 00000100:00100000:0.0:1787784678.693800:0:5794:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00000100:00100000:0.0:1787784678.693807:0:5794:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+3:17504:x1874627668348288:12345-0@lo:104: 00010000:00010000:0.0:1787784678.693817:0:5794:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784678.693870:0:5794:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787784678.693875:0:5794:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+3:17504:x1874627668348288:12345-0@lo:104: Request processed in 69us (30127us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784678.693881:0:5794:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00000100:00100000:2.0:1787784678.693972:0:17504:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021afc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17504:1874627668348288:0@lo:104: 00010000:00010000:2.0:1787784678.694013:0:17504:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.694260:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784678.694266:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.694282:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbcc00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:2.0:1787784678.694286:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.694294:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787784678.694301:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784678.694308:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787784678.694357:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.694365:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787784678.694377:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.694384:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbcc00/0xb0ce63137acc4b55 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4b5c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.694399:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784678.694401:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784678.694403:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784678.697697:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784678.697725:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668349056:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784678.697797:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784678.697987:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349056 00000100:00100000:2.0:1787784678.697995:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349056 00010000:00010000:2.0:1787784678.698030:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 13 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784678.698071:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1787784678.698093:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+13:3279:x1874627668349056:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787784678.698107:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784678.698112:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 13 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698124:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 13 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698140:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032eb600/0xb0ce63137acc4b5c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4b55 expref: 13 pid: 5803 timeout: 619 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698180:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698189:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698205:0:5795:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698212:0:5795:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698218:0:5795:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698224:0:5795:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698231:0:5795:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784678.698235:0:5795:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698249:0:5795:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698265:0:5795:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698276:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784678.698312:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+12:3279:x1874627668349056:12345-0@lo:103:ldlm_bl.0 Request processed in 219us (514us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784678.698319:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00010000:00010000:2.0:1787784678.698362:0:17504:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698371:0:17504:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784678.698378:0:17504:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894010ad000/0xb0ce63137acc4bb0 lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17504 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784678.699168:0:5793:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 0 new 8192-byte reqbufs (1/1 left), rc = 0 00000100:00100000:3.0:1787784678.699474:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668349056:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784679.244192:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784679.244709:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004864 00000100:00100000:0.0:1787784679.244719:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004864 00000100:00100000:0.0:1787784679.244776:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:0.0:1787784679.244784:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:5597:x1874627658004864:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784679.244800:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784679.244819:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0961 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.244868:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.244878:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.244887:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbcc00, rc 0) 00000100:00100000:0.0:1787784679.244944:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658004864:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 161us (798us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.244951:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:1.0:1787784679.262184:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784679.263218:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658004992 00000100:00100000:2.0:1787784679.263229:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658004992 00000100:00100000:2.0:1787784679.263304:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:2.0:1787784679.263323:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658004992:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.263628:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658004992:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 314us (1484us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.263636:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:2.0:1787784679.268041:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784679.272924:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658005120 00000100:00100000:1.0:1787784679.272940:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658005120 00000100:00100000:1.0:1787784679.273008:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 397 00000100:00100000:1.0:1787784679.273019:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658005120:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.273346:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658005120:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 327us (5314us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.273355:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 397 00000100:00100000:3.0:1787784679.277094:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784679.277202:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658005248 00000100:00100000:0.0:1787784679.277211:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658005248 00000100:00100000:0.0:1787784679.277268:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 398 00000100:00100000:0.0:1787784679.277489:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22500:x1874627658005248:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:0.0:1787784679.277785:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22500:x1874627658005248:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 300us (700us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.287158:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 398 00000100:00100000:0.0:1787784679.303831:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784679.303936:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658005376 00000100:00100000:2.0:1787784679.303958:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658005376 00000100:00100000:2.0:1787784679.304014:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 399 00000100:00100000:2.0:1787784679.304024:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22500:x1874627658005376:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784679.304292:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22500:x1874627658005376:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 268us (470us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.304300:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 399 00000100:00100000:2.0:1787784679.329557:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784679.330034:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658005504 00000100:00100000:0.0:1787784679.330045:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658005504 00000100:00100000:0.0:1787784679.330098:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:0.0:1787784679.330107:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22500:x1874627658005504:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:0.0:1787784679.330387:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:22500:x1874627658005504:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 279us (839us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.330397:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:3.0:1787784679.333429:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784679.333595:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658005632 00000100:00100000:1.0:1787784679.333606:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658005632 00000100:00100000:1.0:1787784679.333661:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:1.0:1787784679.333677:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658005632:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784679.333696:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784679.333714:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0968 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.333740:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.333768:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.333778:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351800, rc 0) 00000100:00100000:1.0:1787784679.333836:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658005632:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 165us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.333843:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:1.0:1787784679.340758:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784679.340857:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658005760 00000100:00100000:2.0:1787784679.340867:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658005760 00000100:00100000:2.0:1787784679.340931:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 402 00000100:00100000:2.0:1787784679.340941:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658005760:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.341247:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658005760:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 306us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.341255:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 402 00010000:00010000:2.0:1787784679.346499:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784679.349659:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784679.349804:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658005888 00000100:00100000:0.0:1787784679.349815:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658005888 00000100:00100000:0.0:1787784679.349869:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 403 00000100:00100000:0.0:1787784679.349879:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658005888:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.350110:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658005888:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 231us (459us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.350118:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 403 00000100:00100000:3.0:1787784679.353850:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784679.354052:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006016 00000100:00100000:1.0:1787784679.354061:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006016 00000100:00100000:1.0:1787784679.354116:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00100000:1.0:1787784679.354126:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22501:x1874627658006016:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:1.0:1787784679.354559:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22501:x1874627658006016:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 432us (717us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.354569:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00100000:1.0:1787784679.361542:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784679.362016:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006144 00000100:00100000:0.0:1787784679.362026:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006144 00000100:00100000:0.0:1787784679.362124:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:0.0:1787784679.362134:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22501:x1874627658006144:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:0.0:1787784679.362459:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22501:x1874627658006144:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 326us (1072us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.362467:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:3.0:1787784679.369869:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784679.370243:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006272 00000100:00100000:2.0:1787784679.370253:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006272 00000100:00100000:2.0:1787784679.370343:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:2.0:1787784679.370354:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22501:x1874627658006272:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784679.370867:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:22501:x1874627658006272:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 513us (1005us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.370876:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:3.0:1787784679.382037:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784679.382229:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006400 00000100:00100000:2.0:1787784679.382238:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006400 00000100:00100000:2.0:1787784679.382292:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 407 00000100:00100000:2.0:1787784679.382325:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658006400:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784679.382543:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784679.382561:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b096f expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.382589:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.382600:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.382610:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb200, rc 0) 00000100:00100000:2.0:1787784679.382715:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658006400:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 392us (689us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.382724:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 407 00000100:00100000:1.0:1787784679.390965:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784679.391059:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006528 00000100:00100000:0.0:1787784679.391069:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006528 00000100:00100000:0.0:1787784679.391123:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 408 00000100:00100000:0.0:1787784679.391133:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658006528:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.391469:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658006528:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 336us (512us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784679.391479:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 408 10000000:01000000:0.0:1787784679.858183:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787784679.858191:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989518f73800 from 45 10000000:01000000:0.0:1787784679.858196:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787784679.858244:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.858275:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.858287:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784679.858313:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305143c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349184:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.862259:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.862335:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce840 going to sleep for 11 seconds 00000100:00100000:2.0:1787784679.863556:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349184 00000100:00100000:2.0:1787784679.863568:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349184 00000100:00100000:2.0:1787784679.863634:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1787784679.863644:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+12:5808:x1874627668349184:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784679.863661:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784679.863680:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4bcc expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.863707:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.863718:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.863746:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea800, rc 0) 00000100:00100000:2.0:1787784679.863830:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668349184:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 187us (1576us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.863837:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:3.0:1787784679.864088:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305143c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349184:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784679.864103:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.864111:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784679.864217:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349312:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.864252:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784679.864287:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.864522:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349312 00000100:00100000:1.0:1787784679.864528:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349312 00000100:00100000:1.0:1787784679.864563:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1787784679.864571:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668349312:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.864833:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668349312:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 263us (580us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.864841:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:3.0:1787784679.865500:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349312:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.865542:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349440:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.865590:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784679.865627:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.865703:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349440 00000100:00100000:0.0:1787784679.865719:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349440 00000100:00100000:0.0:1787784679.865755:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1787784679.865770:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668349440:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.865959:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668349440:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 189us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.865968:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:3.0:1787784679.866310:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349440:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.867558:0:17506:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17506:1874627668349568:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.867624:0:17506:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.867665:0:17506:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.867725:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349568 00000100:00100000:1.0:1787784679.867737:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349568 00000100:00100000:1.0:1787784679.867787:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1787784679.867796:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:17506:x1874627668349568:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784679.868042:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:17506:x1874627668349568:12345-0@lo:502:llog_process_th.0 Request processed in 246us (417us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.868050:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1787784679.868347:0:17506:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17506:1874627668349568:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.868394:0:17506:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17506:1874627668349696:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.868431:0:17506:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.868462:0:17506:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784679.868724:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349696 00000100:00100000:3.0:1787784679.868728:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349696 00000100:00100000:3.0:1787784679.868770:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:3.0:1787784679.868777:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:17506:x1874627668349696:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784679.868929:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:17506:x1874627668349696:12345-0@lo:502:llog_process_th.0 Request processed in 152us (496us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784679.868935:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1787784679.869287:0:17506:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17506:1874627668349696:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.870059:0:17506:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17506:1874627668349824:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.870108:0:17506:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.870187:0:17506:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784679.875384:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349824 00000100:00100000:3.0:1787784679.875399:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349824 00000100:00100000:3.0:1787784679.875467:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1787784679.875477:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:17506:x1874627668349824:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784679.875739:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:17506:x1874627668349824:12345-0@lo:502:llog_process_th.0 Request processed in 262us (5629us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784679.875747:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1787784679.875927:0:17506:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17506:1874627668349824:0@lo:502:llog_process_th.0 00000040:00080000:0.0:1787784679.875965:0:17506:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000020:01000000:3.0:1787784679.877898:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 45-48 (rc=0) 10000000:01000000:3.0:1787784679.877935:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1787784679.877947:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.877958:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784679.877968:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:3.0:1787784679.877971:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:3.0:1787784679.877974:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:3.0:1787784679.878012:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.878030:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.878039:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784679.878077:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349952:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.878135:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784679.878208:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066de40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.878511:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668349952 00000100:00100000:0.0:1787784679.878518:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668349952 00000100:00100000:0.0:1787784679.878563:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:0.0:1787784679.878571:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668349952:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784679.878584:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784679.878597:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4bda expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.878620:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 14 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.878630:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 14 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.878638:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbdc00, rc 0) 00000100:00100000:0.0:1787784679.878676:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668349952:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 106us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.878681:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1787784679.878922:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668349952:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784679.878936:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.878944:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784679.878994:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350080:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.879034:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784679.879062:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180480 going to sleep for 11 seconds 00000100:00100000:3.0:1787784679.879257:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350080 00000100:00100000:3.0:1787784679.879261:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350080 00000100:00100000:3.0:1787784679.879293:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1787784679.879300:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668350080:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.879561:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668350080:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 261us (526us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784679.879569:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:2.0:1787784679.879701:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350080:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.879729:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350208:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.879781:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784679.879806:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180480 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.879903:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350208 00000100:00100000:1.0:1787784679.879908:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350208 00000100:00100000:1.0:1787784679.879929:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:1.0:1787784679.879936:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668350208:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.880091:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668350208:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 155us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.880098:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:2.0:1787784679.880439:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350208:0@lo:503:ll_cfg_requeue.0 00000040:00080000:0.0:1787784679.881737:0:17507:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:2.0:1787784679.883427:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:2.0:1787784679.883457:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:2.0:1787784679.883468:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.883499:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784679.883509:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787784679.883512:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989538a62800 from 45 10000000:01000000:2.0:1787784679.883514:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787784679.883562:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.883571:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.883581:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784679.883605:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350336:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.883669:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784679.883702:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180480 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.883783:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350336 00000100:00100000:1.0:1787784679.883790:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350336 00000100:00100000:1.0:1787784679.883837:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:1.0:1787784679.883845:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668350336:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784679.883858:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784679.883872:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4be8 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.883894:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.883905:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.883914:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403350200, rc 0) 00000100:00100000:1.0:1787784679.883965:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668350336:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 120us (296us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.883972:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:2.0:1787784679.884457:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350336:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784679.884472:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.884481:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784679.884533:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350464:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.884564:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784679.884600:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180480 going to sleep for 11 seconds 00000100:00100000:3.0:1787784679.884650:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350464 00000100:00100000:3.0:1787784679.884655:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350464 00000100:00100000:3.0:1787784679.884688:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:3.0:1787784679.884696:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668350464:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.885536:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668350464:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 837us (968us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784679.885545:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:2.0:1787784679.885639:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350464:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.885677:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350592:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.885717:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784679.885787:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180480 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.886959:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350592 00000100:00100000:0.0:1787784679.886969:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350592 00000100:00100000:0.0:1787784679.890809:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:0.0:1787784679.890852:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668350592:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.892067:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668350592:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 1222us (6345us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.892078:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:2.0:1787784679.896500:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668350592:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.899844:0:17508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17508:1874627668350720:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784679.899932:0:17508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784679.899977:0:17508:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784679.900458:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350720 00000100:00100000:3.0:1787784679.900466:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350720 00000100:00100000:3.0:1787784679.900515:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:3.0:1787784679.900525:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17508:x1874627668350720:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784679.900788:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17508:x1874627668350720:12345-0@lo:502:llog_process_th.0 Request processed in 263us (855us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784679.900796:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:2.0:1787784679.901424:0:17508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17508:1874627668350720:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784679.901478:0:17508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17508:1874627668350848:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784679.901520:0:17508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784679.901588:0:17508:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180300 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.903919:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350848 00000100:00100000:1.0:1787784679.903933:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350848 00000100:00100000:1.0:1787784679.903997:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1787784679.904007:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:17508:x1874627668350848:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784679.904554:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:17508:x1874627668350848:12345-0@lo:502:llog_process_th.0 Request processed in 547us (3032us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.904562:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:0.0:1787784679.904671:0:17508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17508:1874627668350848:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.904727:0:17508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17508:1874627668350976:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.904771:0:17508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.904807:0:17508:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a5ccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.904906:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668350976 00000100:00100000:1.0:1787784679.904911:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668350976 00000100:00100000:1.0:1787784679.904944:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:1.0:1787784679.904951:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:17508:x1874627668350976:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784679.905085:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:17508:x1874627668350976:12345-0@lo:502:llog_process_th.0 Request processed in 134us (314us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.905093:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:2.0:1787784679.905287:0:17508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17508:1874627668350976:0@lo:502:llog_process_th.0 00000040:00080000:2.0:1787784679.905313:0:17508:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000020:01000000:1.0:1787784679.905694:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 45-48 (rc=0) 10000000:01000000:1.0:1787784679.905711:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1787784679.905721:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.905732:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784679.905741:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:1.0:1787784679.905744:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:1.0:1787784679.905747:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:1.0:1787784679.905782:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.905791:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.905799:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784679.905814:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351104:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.905846:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.905869:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784679.905918:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668351104 00000100:00100000:2.0:1787784679.905922:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668351104 00000100:00100000:2.0:1787784679.905955:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:2.0:1787784679.905962:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668351104:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784679.905971:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784679.905983:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4bf6 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.906004:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.906036:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.906045:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eaa00, rc 0) 00000100:00100000:2.0:1787784679.906086:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668351104:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 124us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.906093:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:1.0:1787784679.906286:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351104:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784679.906299:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.906308:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784679.906363:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351232:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.906387:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.906406:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784679.906512:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668351232 00000100:00100000:2.0:1787784679.906516:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668351232 00000100:00100000:2.0:1787784679.906533:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:2.0:1787784679.906539:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668351232:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784679.906755:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668351232:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 215us (367us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.906762:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:1.0:1787784679.906930:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351232:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.906959:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351360:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.906997:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.907028:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784679.907128:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668351360 00000100:00100000:2.0:1787784679.907132:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668351360 00000100:00100000:2.0:1787784679.907180:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:2.0:1787784679.907186:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668351360:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.907508:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351360:0@lo:503:ll_cfg_requeue.0 00000040:00080000:1.0:1787784679.908514:0:17509:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000100:00100000:2.0:1787784679.908779:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668351360:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 136us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.908791:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000020:01000000:3.0:1787784679.914480:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:3.0:1787784679.914529:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:3.0:1787784679.914540:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.914553:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784679.914563:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787784679.914566:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9895054dd800 from 45 10000000:01000000:3.0:1787784679.914569:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787784679.914606:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.914615:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.914626:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784679.914650:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351488:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.914716:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784679.914772:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784679.914875:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668351488 00000100:00100000:2.0:1787784679.914882:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668351488 00000100:00100000:2.0:1787784679.914943:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:2.0:1787784679.914952:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668351488:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784679.914965:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784679.914986:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4c04 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.915009:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.915030:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784679.915040:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eba00, rc 0) 00000100:00100000:2.0:1787784679.915087:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668351488:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 136us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784679.915094:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1787784679.916662:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351488:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784679.916688:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784679.916700:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784679.916788:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351616:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.916847:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784679.916916:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784679.921649:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784679.921675:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530515e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668351744:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784679.921750:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.921941:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668351744 00000100:00100000:0.0:1787784679.921949:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668351744 00000100:00100000:0.0:1787784679.922016:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:0.0:1787784679.922027:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668351744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784679.922060:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784679.922111:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668351744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (361us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.922118:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00080000:2.0:1787784679.922808:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784679.922848:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530515e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668351744:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784679.924912:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668351616 00000100:00100000:1.0:1787784679.924924:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668351616 00000100:00100000:1.0:1787784679.924975:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1787784679.924984:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668351616:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.925595:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668351616:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 610us (8746us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.925604:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:0.0:1787784679.930484:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351616:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.930664:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351872:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.930745:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.930789:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.930861:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668351872 00000100:00100000:1.0:1787784679.930868:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668351872 00000100:00100000:1.0:1787784679.930964:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:1.0:1787784679.930974:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668351872:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.931218:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668351872:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 244us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.931226:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:0.0:1787784679.931726:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668351872:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.932263:0:17510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17510:1874627668352000:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.932314:0:17510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.943375:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352000 00000100:00100000:1.0:1787784679.943390:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352000 00000100:00100000:1.0:1787784679.943437:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:1.0:1787784679.943447:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17510:x1874627668352000:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.943452:0:17510:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e540 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.943736:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17510:x1874627668352000:12345-0@lo:502:llog_process_th.0 Request processed in 290us (11422us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.943744:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:0.0:1787784679.944490:0:17510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17510:1874627668352000:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.944570:0:17510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17510:1874627668352128:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.944616:0:17510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.944644:0:17510:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e540 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.945821:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352128 00000100:00100000:1.0:1787784679.945831:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352128 00000100:00100000:1.0:1787784679.945873:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:1.0:1787784679.945883:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17510:x1874627668352128:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784679.946106:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17510:x1874627668352128:12345-0@lo:502:llog_process_th.0 Request processed in 224us (1489us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.946113:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:1.0:1787784679.946200:0:17510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17510:1874627668352128:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784679.946258:0:17510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17510:1874627668352256:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784679.946298:0:17510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.946339:0:17510:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405d8780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.946348:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352256 00000100:00100000:0.0:1787784679.946353:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352256 00000100:00100000:0.0:1787784679.946372:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:0.0:1787784679.946378:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17510:x1874627668352256:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.946494:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17510:x1874627668352256:12345-0@lo:502:llog_process_th.0 Request processed in 114us (194us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.946501:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1787784679.946567:0:17510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17510:1874627668352256:0@lo:502:llog_process_th.0 00000040:00080000:1.0:1787784679.946589:0:17510:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000020:01000000:0.0:1787784679.946681:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 45-48 (rc=0) 10000000:01000000:0.0:1787784679.946699:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787784679.946709:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.946719:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784679.946729:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:0.0:1787784679.946731:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 40 10000000:01000000:0.0:1787784679.946734:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:0.0:1787784679.946756:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.946764:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.946784:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784679.946796:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668352384:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.946824:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.946845:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.947371:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352384 00000100:00100000:1.0:1787784679.947376:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352384 00000100:00100000:1.0:1787784679.947401:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:1.0:1787784679.947408:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668352384:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784679.947417:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784679.947429:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4c12 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.948119:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.948131:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.948140:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:1.0:1787784679.948430:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668352384:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 1021us (1603us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.948451:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1787784679.959352:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668352384:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784679.959383:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.959397:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784679.960658:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668352512:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.960721:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784679.960754:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352512 00000100:00100000:0.0:1787784679.960760:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352512 00000100:00100000:1.0:1787784679.960762:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405d8d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.960798:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1787784679.960805:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668352512:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784679.961131:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668352512:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 324us (409us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.961140:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:1.0:1787784679.961203:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668352512:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.961236:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668352640:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.961274:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.961293:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405d8d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784679.961306:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352640 00000100:00100000:3.0:1787784679.961312:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352640 00000100:00100000:3.0:1787784679.961351:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:3.0:1787784679.961360:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668352640:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.961580:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668352640:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 220us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784679.961590:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668352640:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784679.961591:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:3.0:1787784679.962115:0:17511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17511:1874627668352768:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784679.962179:0:17511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784679.962201:0:17511:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066da80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.962470:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352768 00000100:00100000:0.0:1787784679.962474:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352768 00000100:00100000:0.0:1787784679.962505:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1787784679.962512:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:17511:x1874627668352768:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.962820:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:17511:x1874627668352768:12345-0@lo:502:llog_process_th.0 Request processed in 308us (640us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.962855:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:3.0:1787784679.962930:0:17511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17511:1874627668352768:0@lo:502:llog_process_th.0 00000020:01000000:3.0:1787784679.962958:0:17511:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:3.0:1787784679.962969:0:17511:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1787784679.962971:0:17511:0:(obd_config.c:1563:class_process_config()) marker 31 (0x1) lustre-MDT0000-m add lustre.test_11c.lustre-OST0001_UUID 00000020:00000080:3.0:1787784679.962977:0:17511:0:(obd_config.c:1472:class_process_config()) processing cmd: ce021 00020000:01000000:3.0:1787784679.963064:0:17511:0:(lod_pool.c:594:lod_pool_add()) Added lustre-OST0001_UUID to test_11c as member 1 00000100:00100000:3.0:1787784679.964380:0:17511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17511:1874627668352896:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784679.964440:0:17511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:01000004:1.0:1787784679.964599:0:17512:0:(tgt_mount.c:153:server_get_mount()) get mount ffff9895054dd800 from lustre-MDT0000, refs=4 00000020:01000004:1.0:1787784679.964617:0:17512:0:(obd_mount.c:710:lustre_put_lsi()) put ffff9895054dd800 4 00000100:00100000:3.0:1787784679.964989:0:17511:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066da80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.965390:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668352896 00000100:00100000:0.0:1787784679.965397:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668352896 00000100:00100000:0.0:1787784679.965540:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:0.0:1787784679.965550:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:17511:x1874627668352896:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784679.965787:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:17511:x1874627668352896:12345-0@lo:502:llog_process_th.0 Request processed in 238us (1346us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.965795:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1787784679.965908:0:17511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17511:1874627668352896:0@lo:502:llog_process_th.0 00000020:01000000:3.0:1787784679.965935:0:17511:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:3.0:1787784679.965941:0:17511:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1787784679.965943:0:17511:0:(obd_config.c:1563:class_process_config()) marker 31 (0x2) lustre-MDT0000-m add lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:3.0:1787784679.965948:0:17511:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 43 count 43 00000020:01000000:1.0:1787784679.966381:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 40-42 (rc=0) 10000000:01000000:1.0:1787784679.966401:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:1.0:1787784679.966411:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.966423:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784679.966433:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787784679.966436:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff989508964d28 from 1 10000000:01000000:1.0:1787784679.966438:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787784679.966464:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.966561:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.966578:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784679.966595:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668353024:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.966702:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.966778:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405d8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784679.966866:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668353024 00000100:00100000:0.0:1787784679.966871:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668353024 00000100:00100000:0.0:1787784679.966906:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:0.0:1787784679.966914:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668353024:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784679.966924:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784679.966937:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4c20 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.966958:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 19 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.966969:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 19 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784679.966978:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784679.967019:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668353024:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 106us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784679.967026:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:1.0:1787784679.967193:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668353024:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784679.967206:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.967225:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784679.967267:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668353152:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.967291:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784679.967307:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405d8a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784679.967337:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668353152 00000100:00100000:1.0:1787784679.967341:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668353152 00000100:00100000:1.0:1787784679.967359:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:1.0:1787784679.967365:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668353152:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784679.967679:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668353152:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 314us (386us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784679.967687:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:1.0:1787784679.967716:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668353152:0@lo:501:ll_cfg_requeue.0 10000000:01000000:1.0:1787784679.967736:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1787784679.967747:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784679.967754:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784680.201823:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.201875:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.201907:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.201933:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.201950:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.201983:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.202008:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.202036:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658007040 00000100:00100000:3.0:1787784680.202045:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658007040 00000100:00100000:3.0:1787784680.202099:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658007296 00000100:00100000:3.0:1787784680.202103:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658007296 00000100:00100000:3.0:1787784680.202116:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006912 00000100:00100000:3.0:1787784680.202118:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006912 00000100:00100000:2.0:1787784680.202244:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658007424 00000100:00100000:2.0:1787784680.202248:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658007424 00000100:00100000:2.0:1787784680.202275:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 233 00000100:00100000:2.0:1787784680.202283:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2355:x1874627658007040:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784680.202366:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2355:x1874627658007040:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 83us (490us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784680.202375:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 233 00000100:00100000:2.0:1787784680.202391:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 234 00000100:00100000:2.0:1787784680.202398:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658007296:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784680.202438:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658007296:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (530us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784680.202444:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 234 00000100:00100000:2.0:1787784680.202455:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 235 00000100:00100000:2.0:1787784680.202460:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658007424:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784680.202481:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658007424:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (473us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784680.202488:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 235 00000100:00100000:2.0:1787784680.202626:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006784 00000100:00100000:2.0:1787784680.202630:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006784 00000100:00100000:2.0:1787784680.202655:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658007168 00000100:00100000:2.0:1787784680.202657:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658007168 00000100:00100000:2.0:1787784680.202676:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:2.0:1787784680.202682:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:2355:x1874627658006784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784680.202713:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:2355:x1874627658006784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (901us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784680.202720:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:2.0:1787784680.202730:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:2.0:1787784680.202735:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2355:x1874627658007168:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784680.202754:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2355:x1874627658007168:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (771us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784680.202759:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00080000:2.0:1787784680.202782:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784680.202787:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784680.202789:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784680.202982:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 236 00000100:00100000:3.0:1787784680.202990:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2355:x1874627658006912:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784680.203022:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2355:x1874627658006912:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (1071us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784680.203028:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 236 00000100:00100000:1.0:1787784680.206007:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658006656 00000100:00100000:1.0:1787784680.206023:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658006656 00000100:00100000:1.0:1787784680.206092:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:1.0:1787784680.206103:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658006656:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784680.206212:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658006656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 111us (4278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784680.206220:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 440 00010000:00010000:3.0:1787784680.369669:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784680.844946:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784680.845346:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658007680 00000100:00100000:1.0:1787784680.845356:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658007680 00000100:00100000:1.0:1787784680.845420:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 730 00000100:00100000:1.0:1787784680.845430:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:22512:x1874627658007680:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:1.0:1787784680.845495:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784680.845523:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4829 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0849 expref: 73 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.845542:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1300) 00010000:00010000:1.0:1787784680.845547:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784680.845849:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.845863:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 30 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.845882:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.845912:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 30 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.845925:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 30 pid: 10850 timeout: 571 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784680.845939:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668353280:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784680.846290:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950310d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784680.853490:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784680.853555:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658007808 00000100:00100000:0.0:1787784680.853578:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658007808 00010000:00010000:0.0:1787784680.853617:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 31 pid: 10850 timeout: 571 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784680.853663:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 222 00000100:00100000:0.0:1787784680.853673:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2357:x1874627658007808:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784680.853688:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784680.853695:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 31 pid: 10850 timeout: 571 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784680.853709:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 31 pid: 10850 timeout: 571 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784680.853732:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1300) 00010000:00010000:0.0:1787784680.853738:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784680.853747:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784680.853754:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784680.853772:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784680.853824:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2357:x1874627658007808:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 152us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784680.853834:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 222 00010000:00010000:1.0:1787784680.853835:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4868 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b085e expref: 31 pid: 10850 timeout: 571 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784680.853855:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668353280:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784680.853876:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.853885:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.853890:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.853911:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.853920:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.853934:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784680.855658:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 512 00010000:00010000:1.0:1787784680.855673:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.855682:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.855689:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.855694:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784680.855699:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.855707:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.855722:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1300) 00010000:00010000:1.0:1787784680.855725:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784680.855729:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.855735:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c2e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784680.855751:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e33a45e2 reply state 00000000b0c09135 transno 4294967775 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784680.855770:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784680.855870:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:22512:x1874627658007680:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 10438us (10930us total) trans 4294967775 rc 0/0 00000100:00100000:1.0:1787784680.855878:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 730 00010000:00010000:3.0:1787784680.860356:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.860372:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.860380:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.860386:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784680.860398:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.860407:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.860425:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3c4dff78 (ffff9895029d1c00) 00010000:00010000:3.0:1787784680.860428:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784680.860433:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.860439:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4c35 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784680.884620:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784680.885052:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658007936 00000100:00100000:1.0:1787784680.885062:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658007936 00000100:00100000:1.0:1787784680.885117:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 731 00000100:00100000:1.0:1787784680.885127:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+73:22513:x1874627658007936:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784680.885168:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784680.885191:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c3c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0976 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.885256:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc4c43 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.885271:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc4c43 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.885282:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc4c43 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784680.885355:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784680.885360:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 20 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784680.885384:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc4c43 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.885414:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c3c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0976 expref: 75 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.885446:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc4c43 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0976 expref: 74 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.885456:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:1.0:1787784680.885468:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784680.885472:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641600, rc 0) 00000100:00100000:1.0:1787784680.885538:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:22513:x1874627658007936:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 412us (926us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784680.885546:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 731 00000100:00100000:1.0:1787784680.902444:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.902779:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658008064 00000100:00100000:3.0:1787784680.902789:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658008064 00000100:00100000:3.0:1787784680.902975:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 732 00000100:00100000:3.0:1787784680.902986:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:22513:x1874627658008064:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784680.903005:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784680.903025:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c4a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b097d expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903087:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903101:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903112:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903137:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903174:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903183:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903281:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4c5f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903289:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4c5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903297:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4c5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784680.903343:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 512 00000004:00200000:3.0:1787784680.903347:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784680.903368:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4c5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903380:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903387:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903393:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903409:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784680.903414:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903422:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903435:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:3.0:1787784680.903438:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784680.903442:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903458:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4c51 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903472:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903479:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903485:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903490:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784680.903493:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903499:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903517:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3c4dff78 (ffff989505542600) 00010000:00010000:3.0:1787784680.903519:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784680.903523:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903529:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4c58 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903560:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c4a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b097d expref: 76 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903572:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4c5f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b097d expref: 75 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.903580:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff989505542200) 00010000:00010000:3.0:1787784680.903582:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784680.903584:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2e00, rc 0) 00000100:00100000:3.0:1787784680.903650:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:22513:x1874627658008064:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 666us (1218us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784680.903667:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 732 00000100:00100000:3.0:1787784680.911389:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784680.911983:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658008320 00000100:00100000:1.0:1787784680.911994:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658008320 00000100:00100000:1.0:1787784680.912051:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 733 00000100:00100000:1.0:1787784680.912061:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:22513:x1874627658008320:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784680.912080:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784680.912099:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c66 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0984 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784680.912238:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 512 00010000:00010000:1.0:1787784680.912360:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c66 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0984 expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.913827:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:1.0:1787784680.913834:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784680.913844:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4c66 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0984 expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.914503:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640800, rc 0) 00000100:00100000:1.0:1787784680.914577:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:22513:x1874627658008320:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2515us (3196us total) trans 4294967776 rc 301/301 00000100:00100000:1.0:1787784680.914585:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 733 00000100:00100000:0.0:1787784680.926725:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.926814:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658008576 00000100:00100000:3.0:1787784680.926823:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658008576 00000100:00100000:3.0:1787784680.926882:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 734 00000100:00100000:3.0:1787784680.926891:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:22513:x1874627658008576:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:3.0:1787784680.926952:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784680.926983:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4c5f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b097d expref: 76 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.927017:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff989505542200) 00010000:00010000:3.0:1787784680.927028:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784680.927052:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.927065:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784680.927074:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784680.927380:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001d306db4 reply state 000000009798a6d0 transno 4294967777 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784680.927390:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1787784680.927404:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 512 00000100:00100000:3.0:1787784680.927610:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:22513:x1874627658008576:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 719us (906us total) trans 4294967777 rc 0/0 00000100:00100000:3.0:1787784680.927619:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 734 00010000:00010000:1.0:1787784680.929368:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.929384:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.929391:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.929397:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784680.929402:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.929410:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.929426:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff989505542600) 00010000:00010000:1.0:1787784680.929429:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784680.929434:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784680.929441:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4c74 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784680.930449:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784680.930571:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658008832 00000100:00100000:3.0:1787784680.930582:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658008832 00000100:00100000:3.0:1787784680.930634:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 151 00000100:00100000:3.0:1787784680.930644:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:22513:x1874627658008832:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:3.0:1787784680.931214:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:22513:x1874627658008832:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 570us (769us total) trans 4294967778 rc 0/0 00000100:00100000:3.0:1787784680.931224:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 151 00000100:00100000:3.0:1787784681.354270:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784681.354407:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658009088 00000100:00100000:2.0:1787784681.354423:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658009088 00000100:00100000:2.0:1787784681.354503:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 735 00000100:00100000:2.0:1787784681.354514:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:22550:x1874627658009088:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:2.0:1787784681.355016:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784681.355135:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc4c43 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0976 expref: 77 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.355253:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:2.0:1787784681.355264:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784681.355452:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:22550:x1874627658009088:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 939us (1195us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787784681.355460:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 735 00000100:00100000:0.0:1787784681.360398:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784681.360468:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658009216 00000100:00100000:1.0:1787784681.360496:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658009216 00000100:00100000:1.0:1787784681.360563:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 736 00000100:00100000:1.0:1787784681.360574:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:22550:x1874627658009216:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:1.0:1787784681.360594:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784681.360617:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc4c7b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b098b expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.360679:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4c82 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.360693:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4c82 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.360704:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4c82 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784681.360759:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784681.360763:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 20 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784681.360784:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4c82 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.360806:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc4c7b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b098b expref: 78 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.360825:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4c82 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b098b expref: 77 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.360834:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:1.0:1787784681.360837:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784681.360840:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641000, rc 0) 00000100:00100000:1.0:1787784681.360915:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:22550:x1874627658009216:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 343us (532us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784681.360923:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 736 00000100:00100000:0.0:1787784681.365591:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784681.365666:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658009344 00000100:00100000:1.0:1787784681.365682:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658009344 00000100:00100000:1.0:1787784681.365826:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 737 00000100:00100000:1.0:1787784681.365837:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:22550:x1874627658009344:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:1.0:1787784681.365857:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784681.365876:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc4c89 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0992 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.365935:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.365950:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.365962:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.365980:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.365987:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.365994:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366089:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4c9e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366110:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366117:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784681.366343:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 512 00000004:00200000:1.0:1787784681.366347:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784681.366371:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366384:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366390:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366396:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366401:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784681.366405:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366412:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366423:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:1.0:1787784681.366426:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784681.366431:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366449:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc4c90 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366462:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366468:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366473:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366478:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784681.366482:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366487:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366495:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3c4dff78 (ffff9895029d1e00) 00010000:00010000:1.0:1787784681.366497:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784681.366500:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366505:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc4c97 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366536:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc4c89 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0992 expref: 79 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366548:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4c9e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0992 expref: 78 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784681.366561:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff9895029d1c00) 00010000:00010000:1.0:1787784681.366564:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784681.366566:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640e00, rc 0) 00000100:00100000:1.0:1787784681.366639:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:22550:x1874627658009344:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 805us (1067us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784681.366647:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 737 00010000:00010000:2.0:1787784681.393321:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784681.431003:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784681.435296:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658009472 00000100:00100000:0.0:1787784681.435311:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658009472 00000100:00100000:0.0:1787784681.435871:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 738 00000100:00100000:0.0:1787784681.435883:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+30:22556:x1874627658009472:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784681.435902:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784681.435922:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4ca5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0999 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.435986:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.436001:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.436842:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784681.436910:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784681.436915:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 20 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784681.436942:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.436970:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbde00/0xb0ce63137acc4ca5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 32 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.436989:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 31 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.437693:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:0.0:1787784681.437705:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784681.437709:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd600, rc 0) 00000100:00100000:0.0:1787784681.437802:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:22556:x1874627658009472:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1920us (6807us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784681.437811:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 738 00000100:00100000:0.0:1787784681.451391:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784681.451927:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658009728 00000100:00100000:2.0:1787784681.451937:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658009728 00000100:00100000:2.0:1787784681.452169:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 739 00000100:00100000:2.0:1787784681.452181:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:22556:x1874627658009728:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784681.452201:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784681.452204:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787784681.452230:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4c9e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0992 expref: 78 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.452251:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff9895029d1c00) 00010000:00010000:2.0:1787784681.452263:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784681.452290:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4cb3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09a0 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.452679:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.452692:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.452702:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.452715:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.452722:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.452728:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454243:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454257:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454268:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784681.454278:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2f:0x0]: returning size 0 00010000:00010000:2.0:1787784681.454303:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454334:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454341:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454348:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784681.454352:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454359:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454371:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff989530cb1800) 00010000:00010000:2.0:1787784681.454375:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784681.454379:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454386:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4cba lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784681.454398:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000048da1583 reply state 00000000fd72036e transno 4294967779 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784681.454406:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784681.454439:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea600/0xb0ce63137acc4cb3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 80 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454457:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 79 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.454476:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784681.454478:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784681.454481:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea400, rc 0) 00000100:00100000:2.0:1787784681.454551:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:22556:x1874627658009728:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2372us (3177us total) trans 4294967779 rc 0/0 00000100:00100000:2.0:1787784681.454560:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 739 00010000:00010000:3.0:1787784681.466727:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.466746:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.466754:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.466769:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784681.466775:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.466783:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.466799:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0xecadbcfe (ffff989530cb1700) 00010000:00010000:3.0:1787784681.466803:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784681.466807:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.466814:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cc1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784681.468706:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784681.468984:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658009984 00000100:00100000:3.0:1787784681.468994:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658009984 00000100:00100000:3.0:1787784681.469048:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 740 00000100:00100000:3.0:1787784681.469057:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:22556:x1874627658009984:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784681.469082:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784681.469104:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4cd6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09a7 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.469244:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.469260:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 80 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.469269:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.469302:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 80 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.469327:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 80 pid: 10850 timeout: 572 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784681.469341:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668353408:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784681.469558:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066db40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784681.472437:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668353408:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784681.472465:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784681.473681:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784681.473874:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658010112 00000100:00100000:3.0:1787784681.473884:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658010112 00010000:00010000:3.0:1787784681.473932:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 81 pid: 10850 timeout: 572 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784681.473967:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 223 00000100:00100000:3.0:1787784681.473989:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:2356:x1874627658010112:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784681.474006:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784681.474014:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 81 pid: 10850 timeout: 572 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.474028:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 81 pid: 10850 timeout: 572 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.474051:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea400/0xb0ce63137acc4cc8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a0 expref: 81 pid: 10850 timeout: 572 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.474066:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff989530cb1400) 00010000:00010000:3.0:1787784681.474071:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.474079:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.474087:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.474353:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784681.474413:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:2356:x1874627658010112:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 425us (739us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784681.474422:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 223 00010000:00010000:0.0:1787784681.476009:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476023:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784681.476167:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2f:0x0]: returning size 0 00010000:00010000:0.0:1787784681.476430:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476438:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476445:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476451:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784681.476455:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476463:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476490:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff989530cb1400) 00010000:00010000:0.0:1787784681.476493:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784681.476498:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476504:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4cdd lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476556:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4cd6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a7 expref: 80 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476565:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff989505542200) 00010000:00010000:0.0:1787784681.476568:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784681.476571:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4cd6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09a7 expref: 80 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.476582:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2e00, rc 0) 00000100:00100000:0.0:1787784681.476638:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:22556:x1874627658009984:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 7582us (7951us total) trans 4294967780 rc 301/301 00000100:00100000:0.0:1787784681.476646:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 740 00000100:00100000:1.0:1787784681.481867:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784681.482012:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658010368 00000100:00100000:2.0:1787784681.482023:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658010368 00000100:00100000:2.0:1787784681.482082:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 152 00000100:00100000:2.0:1787784681.482093:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:22556:x1874627658010368:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784681.482689:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:22556:x1874627658010368:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 596us (939us total) trans 4294967781 rc 0/0 00000100:00100000:2.0:1787784681.483651:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 152 00000100:00100000:2.0:1787784681.489735:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784681.490566:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658010496 00000100:00100000:3.0:1787784681.490576:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658010496 00000100:00100000:2.0:1787784681.500867:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 741 00000100:00100000:2.0:1787784681.500881:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:22556:x1874627658010496:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784681.500905:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784681.500928:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4ceb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09ae expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.500992:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4ceb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09ae expref: 81 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.501007:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032eb600/0xb0ce63137acc4ceb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09ae expref: 81 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.501044:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff989530cb1800) 00010000:00010000:2.0:1787784681.501047:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784681.501051:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb600, rc 0) 00000100:00100000:2.0:1787784681.503547:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:22556:x1874627658010496:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2658us (13812us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784681.503560:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 741 00000100:00100000:1.0:1787784681.519759:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784681.520110:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658010752 00000100:00100000:0.0:1787784681.520121:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658010752 00000100:00100000:3.0:1787784681.520339:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 742 00000100:00100000:3.0:1787784681.520352:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:22556:x1874627658010752:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1787784681.520448:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520465:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520476:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520674:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520682:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520688:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520694:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784681.520853:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520863:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520876:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff989530cb1800) 00010000:00010000:3.0:1787784681.520879:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787784681.520883:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784681.520889:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3c00/0xb0ce63137acc4cf2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784681.521034:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:22556:x1874627658010752:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 685us (1346us total) trans 4294967782 rc 0/0 00000100:00100000:3.0:1787784681.521042:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 742 00000100:00100000:3.0:1787784681.523715:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784681.523864:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658011008 00000100:00100000:2.0:1787784681.523874:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658011008 00000100:00100000:2.0:1787784681.523948:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 743 00000100:00100000:2.0:1787784681.523966:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:22556:x1874627658011008:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784681.524259:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.524275:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784681.524284:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784681.524493:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967783 00000004:00010000:2.0:1787784681.524539:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001ebe65aa reply state 0000000042942fd2 transno 4294967784 ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784681.524548:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784681.524628:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:22556:x1874627658011008:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 670us (933us total) trans 4294967784 rc 0/0 00000100:00100000:2.0:1787784681.524635:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 743 00000020:00080000:2.0:1787784681.524653:0:10850:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1787784681.528098:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.528115:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.528123:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.528143:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784681.528187:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.528196:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.528211:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff989530cb1800) 00010000:00010000:0.0:1787784681.528214:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784681.528219:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784681.528224:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894032ea000/0xb0ce63137acc4cf9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784681.529708:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784681.529838:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658011264 00000100:00100000:2.0:1787784681.529848:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658011264 00000100:00100000:2.0:1787784681.529908:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 153 00000100:00100000:2.0:1787784681.529918:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:22556:x1874627658011264:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784681.530141:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:22556:x1874627658011264:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 223us (455us total) trans 4294967785 rc 0/0 00000100:00100000:2.0:1787784681.530180:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 153 00000100:00100000:2.0:1787784681.970489:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668353536:0@lo:400:kworker.0 00000100:00100000:2.0:1787784681.970572:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784681.970627:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668354048:0@lo:400:kworker.0 00000100:00100000:2.0:1787784681.970653:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784681.970676:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354048 00000100:00100000:0.0:1787784681.970684:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354048 00000100:00100000:0.0:1787784681.970749:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1787784681.970769:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668354048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.970847:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668353920:0@lo:400:kworker.0 00000100:00080000:2.0:1787784681.970891:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784681.970930:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668353792:0@lo:400:kworker.0 00000100:00100000:1.0:1787784681.970959:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668353536 00000100:00100000:1.0:1787784681.970963:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668353536 00000100:00100000:1.0:1787784681.971001:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1787784681.971009:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668353536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784681.974586:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668354048:12345-0@lo:400:kworker.0 Request processed in 3819us (3922us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784681.974601:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:3.0:1787784681.975257:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784681.975928:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668353664:0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.975989:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784681.976012:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668354176:0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.976037:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784681.976073:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668353920 00000100:00100000:3.0:1787784681.976081:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668353920 00000100:00100000:3.0:1787784681.976129:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354176 00000100:00100000:3.0:1787784681.976132:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354176 00000100:00100000:3.0:1787784681.976176:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:3.0:1787784681.976186:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668353920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.976240:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668353920:12345-0@lo:400:kworker.0 Request processed in 55us (984us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784681.976247:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:3.0:1787784681.976261:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1787784681.976267:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539378f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668354176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.976296:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539378f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668354176:12345-0@lo:400:kworker.0 Request processed in 30us (259us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784681.976302:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00080000:3.0:1787784681.976341:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784681.976345:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784681.976543:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668353920:0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.976575:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668354176:0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.976594:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668353664 00000100:00100000:3.0:1787784681.976597:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668353664 00000100:00100000:3.0:1787784681.976631:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:3.0:1787784681.976638:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668353664:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.976676:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668353664:12345-0@lo:400:kworker.0 Request processed in 39us (686us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784681.976682:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:3.0:1787784681.976835:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668353664:0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.976880:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668354048:0@lo:400:kworker.0 00000100:00100000:1.0:1787784681.977091:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668353536:12345-0@lo:400:kworker.0 Request processed in 6080us (6517us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784681.977100:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1787784681.977131:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668353536:0@lo:400:kworker.0 00000100:00100000:2.0:1787784681.986426:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784681.986496:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668354304:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784681.986546:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784681.986657:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354304 00000100:00100000:0.0:1787784681.986666:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354304 00000100:00100000:0.0:1787784681.986729:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:0.0:1787784681.986739:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668354304:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784681.986785:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784681.986838:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668354304:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (290us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784681.986860:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1787784681.986950:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668354304:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784681.986988:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668353792 00000100:00100000:3.0:1787784681.986993:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668353792 00000100:00100000:3.0:1787784681.987045:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1787784681.987053:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668353792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784681.987093:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668353792:12345-0@lo:400:kworker.0 Request processed in 41us (674us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784681.987099:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00080000:2.0:1787784681.987820:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784681.987849:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668353792:0@lo:400:kworker.0 00010000:00010000:0.0:1787784682.418501:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784682.565651:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967780 is committed 00000001:00080000:3.0:1787784682.565694:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967785 is committed 00000040:00080000:3.0:1787784682.930244:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784682.930254:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784682.932881:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 17 count 1 00000040:00080000:3.0:1787784682.932901:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784683.447675:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784684.465606:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784685.049464:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539379680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668354432:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784685.049570:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784685.049714:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354432 00000100:00100000:0.0:1787784685.049723:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354432 00000100:00100000:0.0:1787784685.049838:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:0.0:1787784685.049848:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668354432:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784685.049900:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784685.049952:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668354432:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784685.049959:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:0.0:1787784685.050019:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539379680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668354432:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787784685.053841:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784685.324999:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784685.325050:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784685.325093:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784685.325129:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784685.325213:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784685.325233:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784685.325263:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784685.325314:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658011648 00000100:00100000:2.0:1787784685.325326:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658011648 00000100:00100000:2.0:1787784685.325397:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012160 00000100:00100000:2.0:1787784685.325400:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012160 00000100:00100000:2.0:1787784685.325414:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658011776 00000100:00100000:2.0:1787784685.325416:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658011776 00000100:00100000:2.0:1787784685.325427:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012032 00000100:00100000:2.0:1787784685.325430:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012032 00000100:00100000:2.0:1787784685.325445:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00100000:2.0:1787784685.325455:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658011648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784685.325552:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658011648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 98us (500us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784685.325560:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00100000:2.0:1787784685.325578:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:2.0:1787784685.325583:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658012160:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784685.325607:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658012160:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784685.325613:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:2.0:1787784685.325621:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 241 00000100:00100000:2.0:1787784685.325626:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658011776:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784685.325642:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658011776:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 17us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784685.325648:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 241 00000100:00100000:2.0:1787784685.325667:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 242 00000100:00100000:2.0:1787784685.325673:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658012032:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784685.325699:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658012032:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784685.325705:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 242 00000100:00080000:2.0:1787784685.325733:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784685.325737:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784685.325832:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658011392 00000100:00100000:1.0:1787784685.325837:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658011392 00000100:00100000:1.0:1787784685.325880:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 442 00000100:00100000:1.0:1787784685.325887:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658011392:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784685.325918:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658011392:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (934us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784685.325934:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 442 00000100:00100000:1.0:1787784685.325972:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658011904 00000100:00100000:1.0:1787784685.325976:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658011904 00000100:00100000:1.0:1787784685.326009:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658011520 00000100:00100000:1.0:1787784685.326013:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658011520 00000100:00100000:1.0:1787784685.326032:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 295 00000100:00100000:1.0:1787784685.326038:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2357:x1874627658011904:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784685.326072:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2357:x1874627658011904:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (979us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784685.326079:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 295 00000100:00100000:1.0:1787784685.326091:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00100000:1.0:1787784685.326097:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2357:x1874627658011520:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784685.326128:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2357:x1874627658011520:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (914us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784685.326134:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00080000:1.0:1787784685.326179:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784685.489715:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784686.514479:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784687.089528:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030eda40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668354688:0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.089604:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784687.089675:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ecf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668355200:0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.089699:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784687.089729:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784687.089734:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668354816:0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.089756:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784687.089846:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539379680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668354560:0@lo:400:kworker.0 00000100:00100000:1.0:1787784687.089883:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784687.090173:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668355072:0@lo:400:kworker.0 00000100:00100000:1.0:1787784687.090199:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784687.090217:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784687.090223:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668354944:0@lo:400:kworker.0 00000100:00100000:1.0:1787784687.090243:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784687.090269:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354688 00000100:00100000:1.0:1787784687.090285:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354688 00000100:00100000:1.0:1787784687.090339:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354816 00000100:00100000:1.0:1787784687.090342:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354816 00000100:00100000:1.0:1787784687.090361:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355072 00000100:00100000:1.0:1787784687.090363:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355072 00000100:00100000:1.0:1787784687.090378:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1787784687.090386:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668354688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784687.090434:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668354688:12345-0@lo:400:kworker.0 Request processed in 48us (829us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784687.090440:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1787784687.090457:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:1.0:1787784687.090462:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668354816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784687.090486:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668354816:12345-0@lo:400:kworker.0 Request processed in 24us (730us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784687.090491:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:1.0:1787784687.090501:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:1.0:1787784687.090507:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668355072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784687.090532:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668355072:12345-0@lo:400:kworker.0 Request processed in 25us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784687.090537:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00080000:1.0:1787784687.090554:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784687.090578:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668355072:0@lo:400:kworker.0 00000100:00100000:2.0:1787784687.090731:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355200 00000100:00100000:2.0:1787784687.090736:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355200 00000100:00100000:2.0:1787784687.090768:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354944 00000100:00100000:2.0:1787784687.090771:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354944 00000100:00100000:2.0:1787784687.090787:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:2.0:1787784687.090793:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668355200:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784687.090826:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668355200:12345-0@lo:400:kworker.0 Request processed in 34us (1126us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784687.090846:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:2.0:1787784687.090860:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:2.0:1787784687.090866:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668354944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784687.091045:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668354944:12345-0@lo:400:kworker.0 Request processed in 31us (653us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784687.091056:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00080000:2.0:1787784687.091093:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784687.091096:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784687.091578:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668354944:0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.091664:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668354560 00000100:00100000:3.0:1787784687.091670:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668354560 00000100:00100000:3.0:1787784687.091711:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1787784687.091719:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668354560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.091765:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668354560:12345-0@lo:400:kworker.0 Request processed in 47us (1881us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784687.091770:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1787784687.091807:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668354816:0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.091819:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030eda40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668354688:0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.091842:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ecf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668355200:0@lo:400:kworker.0 00000100:00100000:2.0:1787784687.092695:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539379680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668354560:0@lo:400:kworker.0 00000100:00100000:3.0:1787784687.092801:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668355328:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784687.092854:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784687.094799:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355328 00000100:00100000:0.0:1787784687.094813:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355328 00000100:00100000:0.0:1787784687.094876:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:0.0:1787784687.094887:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668355328:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784687.094926:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784687.095005:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668355328:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (2150us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784687.095013:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:0.0:1787784687.095074:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668355328:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1787784687.537512:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784688.563330:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784689.586768:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1787784690.098538:0:17806:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007fffd6a62410 20000000:00000080:1.0:1787784690.098556:0:17806:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff98954029c800 uarg=0000000000000000 00000040:00100000:1.0:1787784690.099594:0:17806:0:(llog.c:1072:llog_reverse_process()) index 48 : idx 48 00000040:00100000:1.0:1787784690.099707:0:17806:0:(llog.c:1072:llog_reverse_process()) index 46 : idx 1 20000000:01000000:1.0:1787784690.099713:0:17806:0:(mgs_llog.c:963:mgs_search_pool_cb()) Matched pool rec 46 cmd:0xce021 lustre.test_11c lustre-OST0001_UUID 20000000:01000000:1.0:1787784690.099728:0:17806:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-client/lustre-clilov/add lustre.test_11c.lustre-OST0001_UUID fl=4 20000000:01000000:3.0:1787784690.100413:0:17807:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 45 marker 30 1->4: lustre-clilov add lustre.test_11c.lustre-OST0001_UUID 20000000:01000000:3.0:1787784690.100545:0:17807:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 48 marker 30 2->4: lustre-clilov add lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:3.0:1787784690.100576:0:17807:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000040:00080000:1.0:1787784690.101048:0:17806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].49, 224 off16832 20000000:01000000:1.0:1787784690.101071:0:17806:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce022 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1787784690.101128:0:17806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].50, 136 off16968 00000040:00080000:1.0:1787784690.101714:0:17806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].51, 224 off17192 20000000:01000000:1.0:1787784690.101735:0:17806:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-MDT0000/lustre-MDT0000-mdtlov/add lustre.test_11c.lustre-OST0001_UUID fl=4 20000000:01000000:2.0:1787784690.102432:0:17808:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 40 marker 31 1->4: lustre-MDT0000-mdtlov add lustre.test_11c.lustre-OST0001_UUID 20000000:01000000:2.0:1787784690.102514:0:17808:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 42 marker 31 2->4: lustre-MDT0000-mdtlov add lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:2.0:1787784690.102546:0:17808:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 43 count 43 00000040:00080000:1.0:1787784690.102960:0:17806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].43, 224 off15872 20000000:01000000:1.0:1787784690.102974:0:17806:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce022 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1787784690.103023:0:17806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].44, 144 off16016 00000040:00080000:1.0:1787784690.103060:0:17806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].45, 224 off16240 10000000:01000000:1.0:1787784690.103073:0:17806:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:1.0:1787784690.103091:0:17806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103109:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103118:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103126:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103134:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103141:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103356:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103363:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103377:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103383:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103389:0:17806:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.103397:0:17806:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.103500:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.103514:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.103527:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6aad00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668355456:0@lo:104: 00000100:00100000:2.0:1787784690.103587:0:17806:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784690.103739:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.103747:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 18 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.103757:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668355584:0@lo:104: 00000100:00100000:2.0:1787784690.103779:0:17806:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784690.103849:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.103858:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.103865:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011eed00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668355712:0@lo:104: 00000100:00100000:2.0:1787784690.103885:0:17806:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784690.103944:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355456 00000100:00100000:3.0:1787784690.103955:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355456 00000100:00100000:3.0:1787784690.104000:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355584 00000100:00100000:3.0:1787784690.104003:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355584 00000100:00100000:3.0:1787784690.104017:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:3.0:1787784690.104025:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17806:x1874627668355456:12345-0@lo:104: 00010000:00010000:3.0:1787784690.104036:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104086:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787784690.104091:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17806:x1874627668355456:12345-0@lo:104: Request processed in 68us (504us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784690.104098:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00000100:00100000:3.0:1787784690.104114:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:3.0:1787784690.104120:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17806:x1874627668355584:12345-0@lo:104: 00010000:00010000:3.0:1787784690.104127:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104173:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:3.0:1787784690.104179:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+3:17806:x1874627668355584:12345-0@lo:104: Request processed in 59us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784690.104185:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00010000:00010000:3.0:1787784690.104200:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787784690.104205:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104213:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350c00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784690.104216:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104234:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784690.104242:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784690.104248:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784690.104344:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104351:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787784690.104369:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104376:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989403350c00/0xb0ce63137acc4c20 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c27 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104388:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784690.104391:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104396:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc600 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784690.104399:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104406:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784690.104412:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784690.104422:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784690.104436:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104442:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787784690.104447:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104454:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4c12 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c19 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104461:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784690.104463:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784690.104464:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784690.104471:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784690.104481:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668355968:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784690.104507:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784690.104528:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784690.104533:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668355840:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784690.104550:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784690.104571:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355968 00000100:00100000:3.0:1787784690.104574:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355968 00010000:00010000:3.0:1787784690.104589:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 19 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784690.104687:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355840 00000100:00100000:3.0:1787784690.104691:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355840 00010000:00010000:3.0:1787784690.104705:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 20 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784690.104735:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1787784690.104741:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+20:3277:x1874627668355968:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784690.104752:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784690.104757:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 20 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104766:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 20 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104781:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784690.104808:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+20:3277:x1874627668355968:12345-0@lo:103:ldlm_bl.0 Request processed in 67us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784690.104814:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1787784690.104825:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:3.0:1787784690.104831:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668355840:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784690.104838:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784690.104843:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 19 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104851:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 19 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.104860:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784690.104881:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668355840:12345-0@lo:103:ldlm_bl.0 Request processed in 50us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784690.104887:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:3.0:1787784690.104918:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668355840:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784690.104934:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668355968:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787784690.105182:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.105193:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 18 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105202:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ecb40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356096:0@lo:104: 00000100:00100000:2.0:1787784690.105225:0:17806:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784690.105245:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.105270:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 18 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105279:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ec780 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356224:0@lo:104: 00000100:00100000:2.0:1787784690.105299:0:17806:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784690.105316:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.105324:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 18 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105332:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed2c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356352:0@lo:104: 00000100:00100000:2.0:1787784690.105349:0:17806:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784690.105362:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.105370:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105377:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ede00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356480:0@lo:104: 00000100:00100000:2.0:1787784690.105404:0:17806:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784690.105421:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.105431:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 12 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105440:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ef480 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356608:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784690.105488:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 12 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.105496:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 12 pid: 5803 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105503:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011efc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356736:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784690.105526:0:17806:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.105533:0:17806:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 12 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105541:0:17806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ef840 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356864:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784690.105557:0:17806:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cf30c0 going to sleep for 11 seconds 00010000:00010000:2.0:1787784690.105570:0:17806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbce00/0xb0ce63137acc4c27 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4c20 expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105581:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6aad00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668355456:0@lo:104: 00010000:00010000:2.0:1787784690.105589:0:17806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989403351a00/0xb0ce63137acc4c19 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4c12 expref: 17 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.105603:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668355584:0@lo:104: 00000100:00100000:0.0:1787784690.105789:0:5794:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787784690.105807:0:5794:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668355712 00000100:00100000:0.0:1787784690.105811:0:5794:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668355712 00000100:00100000:0.0:1787784690.105833:0:5794:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668356096 00000100:00100000:0.0:1787784690.105836:0:5794:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668356096 00000100:00100000:0.0:1787784690.105846:0:5794:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668356224 00000100:00100000:0.0:1787784690.105848:0:5794:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668356224 00000100:00100000:0.0:1787784690.105856:0:5794:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668356352 00000100:00100000:0.0:1787784690.105859:0:5794:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668356352 00000100:00100000:0.0:1787784690.105868:0:5794:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668356480 00000100:00100000:0.0:1787784690.105870:0:5794:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668356480 00000100:00100000:0.0:1787784690.105881:0:5794:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:0.0:1787784690.105887:0:5794:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+7:17806:x1874627668355712:12345-0@lo:104: 00010000:00010000:0.0:1787784690.105896:0:5794:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.105950:0:5794:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787784690.105956:0:5794:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+7:17806:x1874627668355712:12345-0@lo:104: Request processed in 69us (2070us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.105963:0:5794:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:0.0:1787784690.105984:0:5794:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 45 00000100:00100000:0.0:1787784690.105990:0:5794:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+6:17806:x1874627668356096:12345-0@lo:104: 00010000:00010000:0.0:1787784690.105998:0:5794:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106028:0:5794:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:0.0:1787784690.106033:0:5794:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+6:17806:x1874627668356096:12345-0@lo:104: Request processed in 44us (808us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.106039:0:5794:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 45 00000100:00100000:0.0:1787784690.106051:0:5794:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:0.0:1787784690.106056:0:5794:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+5:17806:x1874627668356224:12345-0@lo:104: 00010000:00010000:0.0:1787784690.106069:0:5794:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106095:0:5794:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:0.0:1787784690.106100:0:5794:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+5:17806:x1874627668356224:12345-0@lo:104: Request processed in 44us (800us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.106105:0:5794:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00100000:0.0:1787784690.106118:0:5794:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:0.0:1787784690.106123:0:5794:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+4:17806:x1874627668356352:12345-0@lo:104: 00010000:00010000:0.0:1787784690.106129:0:5794:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106182:0:5794:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to priority blp list, 4 blwis in pool 00000100:00100000:0.0:1787784690.106188:0:5794:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+4:17806:x1874627668356352:12345-0@lo:104: Request processed in 65us (838us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.106193:0:5794:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00000100:00100000:0.0:1787784690.106205:0:5794:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:0.0:1787784690.106210:0:5794:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989534579850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+3:17806:x1874627668356480:12345-0@lo:104: 00010000:00010000:0.0:1787784690.106217:0:5794:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106240:0:5794:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to priority blp list, 5 blwis in pool 00000100:00100000:0.0:1787784690.106245:0:5794:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989534579850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+3:17806:x1874627668356480:12345-0@lo:104: Request processed in 35us (841us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.106266:0:5794:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00010000:00010000:0.0:1787784690.106305:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:0.0:1787784690.106310:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106322:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2400 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784690.106325:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106333:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784690.106340:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784690.106345:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784690.106363:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106369:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784690.106387:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106393:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98953f5b2400/0xb0ce63137acc4c04 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4c0b expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106403:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1787784690.106405:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106411:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351400 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784690.106413:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106429:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784690.106435:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784690.106440:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784690.106453:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106458:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784690.106464:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106470:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989403351400/0xb0ce63137acc4bf6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bfd expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106479:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787784690.106482:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106488:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb400 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784690.106490:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106496:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784690.106503:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784690.106512:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784690.106523:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106529:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784690.106541:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106547:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff9894032eb400/0xb0ce63137acc4be8 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bef expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106556:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787784690.106559:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106565:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784690.106567:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106574:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784690.106580:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784690.106584:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784690.106595:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106600:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784690.106733:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106741:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff98953f5b3a00/0xb0ce63137acc4bda lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4be1 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106751:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784690.106754:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106759:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbca00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:0.0:1787784690.106762:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106769:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787784690.106775:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784690.106780:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787784690.106792:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106797:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787784690.106802:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106808:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4bcc lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4bd3 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.106819:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784690.106825:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784690.106827:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784690.106840:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305170c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668356992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.106873:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784690.106902:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668357504:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.106924:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784690.106941:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784690.106946:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668357120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.106964:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784690.106991:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668356992 00000100:00100000:0.0:1787784690.106995:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668356992 00010000:00010000:0.0:1787784690.107013:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 17 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784690.107031:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668357504 00000100:00100000:0.0:1787784690.107034:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668357504 00010000:00010000:0.0:1787784690.107043:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784690.107052:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668357120 00000100:00100000:0.0:1787784690.107054:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668357120 00010000:00010000:0.0:1787784690.107062:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 19 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784690.107073:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:0.0:1787784690.107079:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668356992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787784690.107088:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784690.107093:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 19 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.107113:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 19 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.107127:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784690.107172:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668356992:12345-0@lo:103:ldlm_bl.0 Request processed in 92us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.107180:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:0.0:1787784690.107194:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:0.0:1787784690.107200:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ee050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+18:3278:x1874627668357504:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787784690.107207:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784690.107211:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.107220:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.107229:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784690.107363:0:17806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4c0b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4c04 expref: 18 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.107375:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011eed00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668355712:0@lo:104: 00000100:00100000:2.0:1787784690.107384:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ecb40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356096:0@lo:104: 00000100:00100000:2.0:1787784690.107392:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ec780 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356224:0@lo:104: 00000100:00100000:2.0:1787784690.107400:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed2c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356352:0@lo:104: 00010000:00010000:2.0:1787784690.107409:0:17806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4bd3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4bcc expref: 17 pid: 5804 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.107417:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ede00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356480:0@lo:104: 00000100:00100000:0.0:1787784690.107450:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ee050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+18:3278:x1874627668357504:12345-0@lo:103:ldlm_bl.0 Request processed in 47us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.107459:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:0.0:1787784690.107503:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:0.0:1787784690.107509:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+15:3278:x1874627668357120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787784690.107517:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784690.107521:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 15 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.107530:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 15 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.107546:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4bfd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4bf6 expref: 15 pid: 5805 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.107554:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784690.107576:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+14:3278:x1874627668357120:12345-0@lo:103:ldlm_bl.0 Request processed in 67us (611us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.107582:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:0.0:1787784690.107605:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668357248:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.107773:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784690.107828:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784690.107834:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668357376:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.107855:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787784690.108239:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784690.108529:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668357120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.108539:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305170c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668356992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.108547:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668357504:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784690.108683:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668357248 00000100:00100000:3.0:1787784690.108691:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668357248 00010000:00010000:3.0:1787784690.108717:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 14 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784690.108734:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668357376 00000100:00100000:3.0:1787784690.108736:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668357376 00010000:00010000:3.0:1787784690.108746:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 15 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784690.108756:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:3.0:1787784690.108762:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+15:3276:x1874627668357248:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784690.108770:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784690.108774:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 15 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.108782:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 15 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.108795:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989403350200/0xb0ce63137acc4bef lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4be8 expref: 15 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.108804:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784690.108845:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+14:3276:x1874627668357248:12345-0@lo:103:ldlm_bl.0 Request processed in 82us (1071us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784690.108851:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:3.0:1787784690.108864:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:3.0:1787784690.108870:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+13:3276:x1874627668357376:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784690.108887:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784690.108891:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 13 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.108900:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 13 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.108909:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbdc00/0xb0ce63137acc4be1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4bda expref: 13 pid: 5803 timeout: 631 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784690.108917:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784690.108938:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a5a93c05-6249-4607-a205-909a326a8fd1+12:3276:x1874627668357376:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (1083us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784690.108943:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:0.0:1787784690.109021:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668357376:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.109029:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668357248:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784690.113551:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ef480 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356608:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784690.113564:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011efc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356736:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784690.117394:0:17806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ef840 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17806:1874627668356864:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784690.117452:0:17806:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784690.117795:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784690.117829:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784690.121762:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012288 00000100:00100000:3.0:1787784690.121778:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012288 00000100:00100000:2.0:1787784690.121902:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012544 00000100:00100000:2.0:1787784690.121908:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012544 00010000:00010000:2.0:1787784690.121930:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 14 pid: 5803 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784690.122097:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 231 00000100:00100000:2.0:1787784690.122108:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:2356:x1874627658012544:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784690.122124:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784690.122131:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 14 pid: 5803 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.122142:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 14 pid: 5803 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.122193:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989403351800/0xb0ce63137acc4bbe lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0968 expref: 14 pid: 5803 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784690.122208:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784690.122319:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658012544:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 213us (4488us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784690.122328:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 231 00010000:00010000:3.0:1787784690.122632:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 12 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784690.122691:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 232 00000100:00100000:1.0:1787784690.122702:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:2356:x1874627658012288:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784690.122712:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784690.122717:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 12 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.122726:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 12 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.122738:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4bc5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b096f expref: 12 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.122747:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784690.122783:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:2356:x1874627658012288:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 83us (4989us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784690.122798:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 232 00000100:00100000:0.0:1787784690.122889:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784690.123091:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012416 00000100:00100000:1.0:1787784690.123097:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012416 00010000:00010000:1.0:1787784690.123116:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 11 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784690.123134:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 233 00000100:00100000:1.0:1787784690.123142:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:2356:x1874627658012416:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784690.123165:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784690.123169:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 11 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123178:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 11 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123188:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4bb7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0961 expref: 11 pid: 5804 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123199:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123207:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123220:0:5797:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123226:0:5797:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123243:0:5797:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123263:0:5797:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123270:0:5797:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784690.123274:0:5797:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123283:0:5797:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123299:0:5797:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784690.123310:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784690.123332:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:2356:x1874627658012416:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 191us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784690.123339:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 233 00010000:00010000:0.0:1787784690.123534:0:17806:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.123543:0:17806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784690.123550:0:17806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989403640600/0xb0ce63137acc4d00 lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784690.161418:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668357632:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784690.161487:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784690.161571:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668357632 00000100:00100000:1.0:1787784690.161579:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668357632 00000100:00100000:1.0:1787784690.161645:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:1.0:1787784690.161657:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668357632:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784690.161689:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00080000:2.0:1787784690.161762:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784690.175263:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668357632:12345-0@lo:13:osp-pre-0-0.0 Request processed in 13588us (13755us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784690.175278:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00100000:0.0:1787784690.175443:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668357632:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784690.440021:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784690.440063:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784690.440090:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784690.440120:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784690.440175:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784690.440202:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784690.440231:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012928 00000100:00100000:0.0:1787784690.440241:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012928 00000100:00100000:0.0:1787784690.440307:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013184 00000100:00100000:0.0:1787784690.440311:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013184 00000100:00100000:0.0:1787784690.440326:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012800 00000100:00100000:0.0:1787784690.440328:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012800 00000100:00100000:0.0:1787784690.440339:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013312 00000100:00100000:0.0:1787784690.440341:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013312 00000100:00100000:0.0:1787784690.440356:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:0.0:1787784690.440379:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305170c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658012928:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784690.440450:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305170c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658012928:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 72us (385us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.440457:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:0.0:1787784690.440478:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 246 00000100:00100000:0.0:1787784690.440484:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658013184:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784690.440508:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658013184:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (418us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.440514:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 246 00000100:00100000:0.0:1787784690.440525:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 247 00000100:00100000:0.0:1787784690.440531:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658012800:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784690.440552:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658012800:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.440558:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 247 00000100:00100000:0.0:1787784690.440569:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 248 00000100:00100000:0.0:1787784690.440575:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658013312:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784690.440596:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658013312:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.440602:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 248 00000100:00100000:0.0:1787784690.440632:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658012672 00000100:00100000:0.0:1787784690.440636:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658012672 00000100:00100000:0.0:1787784690.440662:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013056 00000100:00100000:0.0:1787784690.440666:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013056 00000100:00100000:0.0:1787784690.440687:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:0.0:1787784690.440703:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2357:x1874627658012672:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784690.440732:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2357:x1874627658012672:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (721us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.440738:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:0.0:1787784690.440751:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 301 00000100:00100000:0.0:1787784690.440757:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2357:x1874627658013056:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784690.440779:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2357:x1874627658013056:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (632us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784690.440784:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 301 00000100:00080000:0.0:1787784690.440806:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784690.440810:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784690.440811:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787784690.609517:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787784691.505671:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787784691.505680:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989518f73800 from 49 10000000:01000000:1.0:1787784691.505687:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787784691.505736:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.505748:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.505761:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784691.505788:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668357760:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.505858:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.505945:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.506179:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668357760 00000100:00100000:2.0:1787784691.506187:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668357760 00000100:00100000:2.0:1787784691.506255:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:2.0:1787784691.506266:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+12:5808:x1874627668357760:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784691.506281:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784691.506296:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d07 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.506319:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.506330:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.506337:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea800, rc 0) 00000100:00100000:2.0:1787784691.506388:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668357760:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 124us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.506395:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1787784691.506891:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668357760:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784691.506908:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.506918:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784691.506976:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668357888:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.507007:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.507039:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.508632:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668357888 00000100:00100000:2.0:1787784691.508669:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668357888 00000100:00100000:2.0:1787784691.508723:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:2.0:1787784691.508732:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668357888:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.509022:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668357888:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 289us (2012us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.509030:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1787784691.509367:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668357888:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.509430:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358016:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.509477:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.509516:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.509845:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358016 00000100:00100000:2.0:1787784691.509851:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358016 00000100:00100000:2.0:1787784691.509888:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1787784691.509896:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668358016:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.510297:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668358016:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 400us (818us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.510304:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:3.0:1787784691.516639:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358016:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.519389:0:17809:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17809:1874627668358144:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784691.519493:0:17809:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.519538:0:17809:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d000 going to sleep for 11 seconds 00000100:00100000:3.0:1787784691.519819:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358144 00000100:00100000:3.0:1787784691.519826:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358144 00000100:00100000:3.0:1787784691.519874:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:3.0:1787784691.519883:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:17809:x1874627668358144:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784691.520194:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:17809:x1874627668358144:12345-0@lo:502:llog_process_th.0 Request processed in 311us (700us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784691.520203:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1787784691.520772:0:17809:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17809:1874627668358144:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784691.520832:0:17809:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17809:1874627668358272:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784691.520882:0:17809:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784691.520916:0:17809:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401800c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.521026:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358272 00000100:00100000:2.0:1787784691.521031:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358272 00000100:00100000:2.0:1787784691.521067:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1787784691.521075:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:17809:x1874627668358272:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.521258:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:17809:x1874627668358272:12345-0@lo:502:llog_process_th.0 Request processed in 183us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.521266:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:0.0:1787784691.521385:0:17809:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17809:1874627668358272:0@lo:502:llog_process_th.0 00000040:00080000:0.0:1787784691.521428:0:17809:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000020:01000000:2.0:1787784691.521751:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 49-51 (rc=0) 10000000:01000000:2.0:1787784691.521771:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1787784691.521780:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.521790:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784691.521809:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:2.0:1787784691.521812:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:2.0:1787784691.521814:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:2.0:1787784691.521835:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.521842:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.521850:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784691.521861:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358400:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.521886:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.521913:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cf3240 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.522252:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358400 00000100:00100000:1.0:1787784691.522258:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358400 00000100:00100000:1.0:1787784691.522284:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1787784691.522292:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668358400:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784691.522301:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784691.522314:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d15 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.522336:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 14 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.522346:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 14 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.522354:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640600, rc 0) 00000100:00100000:1.0:1787784691.522390:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668358400:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 99us (503us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.522629:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1787784691.522667:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358400:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784691.522680:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.522690:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784691.522751:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358528:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.522777:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.522797:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066df00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.522809:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358528 00000100:00100000:1.0:1787784691.522812:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358528 00000100:00100000:1.0:1787784691.522828:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1787784691.522834:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668358528:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.523047:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668358528:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 212us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.523055:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1787784691.523086:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358528:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.523112:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358656:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.523177:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.523194:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066df00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.523204:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358656 00000100:00100000:1.0:1787784691.523207:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358656 00000100:00100000:1.0:1787784691.523223:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:1.0:1787784691.523228:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668358656:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.528214:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668358656:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 4974us (5054us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.528229:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:0.0:1787784691.528313:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358656:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.529002:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784691.529039:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013440 00000100:00100000:0.0:1787784691.529058:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013440 00000100:00100000:0.0:1787784691.529260:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:0.0:1787784691.529270:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:5597:x1874627658013440:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784691.529292:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784691.529310:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09b5 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.529494:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.529508:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.529708:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc600, rc 0) 00000100:00100000:0.0:1787784691.529768:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658013440:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 498us (768us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784691.529776:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 452 00000040:00080000:2.0:1787784691.530028:0:17810:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:0.0:1787784691.530888:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:0.0:1787784691.530911:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:0.0:1787784691.530920:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.530932:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784691.530942:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787784691.530944:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989538a62800 from 49 10000000:01000000:0.0:1787784691.530946:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787784691.530974:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.530982:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.531002:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784691.531024:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358784:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.531066:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784691.531100:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784691.531169:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358784 00000100:00100000:3.0:1787784691.531179:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358784 00000100:00100000:3.0:1787784691.531211:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:3.0:1787784691.531218:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668358784:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784691.531228:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784691.531238:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d2a expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784691.531269:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784691.531278:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784691.531284:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3c00, rc 0) 00000100:00100000:3.0:1787784691.531319:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668358784:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 100us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784691.531325:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1787784691.532071:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358784:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784691.532088:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.532098:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784691.532200:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358912:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.532240:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784691.532287:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066df00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.532776:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668358912 00000100:00100000:1.0:1787784691.532783:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668358912 00000100:00100000:1.0:1787784691.532824:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:1.0:1787784691.532842:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668358912:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.533096:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668358912:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 255us (854us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.533104:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1787784691.533235:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668358912:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.533283:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359040:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.533326:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784691.534021:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784691.534184:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359040 00000100:00100000:3.0:1787784691.534192:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359040 00000100:00100000:3.0:1787784691.534253:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:3.0:1787784691.534263:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668359040:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784691.535085:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668359040:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 821us (1755us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784691.535095:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:0.0:1787784691.535235:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784691.538294:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013568 00000100:00100000:2.0:1787784691.538309:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013568 00000100:00100000:2.0:1787784691.538366:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 456 00000100:00100000:2.0:1787784691.538375:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658013568:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.538945:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658013568:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 569us (3710us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.538954:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 456 00000100:00100000:1.0:1787784691.540427:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359040:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784691.541024:0:17811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17811:1874627668359168:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784691.541099:0:17811:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784691.541143:0:17811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518ed6000 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.541546:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359168 00000100:00100000:1.0:1787784691.541553:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359168 00000100:00100000:1.0:1787784691.541640:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1787784691.541650:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:17811:x1874627668359168:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784691.541905:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:17811:x1874627668359168:12345-0@lo:502:llog_process_th.0 Request processed in 256us (805us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.541913:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:3.0:1787784691.541988:0:17811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17811:1874627668359168:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784691.542038:0:17811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17811:1874627668359296:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784691.542068:0:17811:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784691.551697:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784691.555779:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359296 00000100:00100000:2.0:1787784691.555798:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359296 00000100:00100000:2.0:1787784691.555862:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:2.0:1787784691.555886:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17811:x1874627668359296:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.556129:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17811:x1874627668359296:12345-0@lo:502:llog_process_th.0 Request processed in 245us (14059us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.556137:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1787784691.558251:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013696 00000100:00100000:1.0:1787784691.558263:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013696 00000100:00100000:1.0:1787784691.558321:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:1.0:1787784691.558331:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658013696:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.559101:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658013696:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 768us (7407us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.559111:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:3.0:1787784691.559909:0:17811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518ed6000 going to sleep for 11 seconds 00000100:00100000:3.0:1787784691.559941:0:17811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17811:1874627668359296:0@lo:502:llog_process_th.0 00000040:00080000:3.0:1787784691.563138:0:17811:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000100:00100000:0.0:1787784691.564067:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784691.564138:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013824 00000100:00100000:2.0:1787784691.564170:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013824 00000100:00100000:2.0:1787784691.564221:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:2.0:1787784691.564232:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23020:x1874627658013824:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784691.564755:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23020:x1874627658013824:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 524us (692us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.564764:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:0.0:1787784691.567258:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784691.567353:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658013952 00000100:00100000:3.0:1787784691.567364:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658013952 00000100:00100000:2.0:1787784691.567653:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 461 00000100:00100000:2.0:1787784691.567665:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23020:x1874627658013952:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784691.567891:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23020:x1874627658013952:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 228us (651us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.567900:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 461 00000020:01000000:1.0:1787784691.571275:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 49-51 (rc=0) 10000000:01000000:1.0:1787784691.571314:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1787784691.571326:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.571339:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784691.571558:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:0.0:1787784691.571627:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:0.0:1787784691.571638:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:0.0:1787784691.571680:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.571689:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.571701:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784691.571727:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359424:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.571793:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784691.571831:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784691.572281:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359424 00000100:00100000:3.0:1787784691.572297:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359424 00000100:00100000:3.0:1787784691.572368:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1787784691.572381:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668359424:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784691.572402:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784691.572423:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d38 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784691.572455:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784691.572467:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784691.572478:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784691.572572:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668359424:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 194us (779us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784691.572581:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1787784691.572712:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359424:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784691.572734:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.572746:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784691.572809:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359552:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.572849:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.572886:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784691.573257:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359552 00000100:00100000:3.0:1787784691.573276:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359552 00000100:00100000:3.0:1787784691.573321:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1787784691.573330:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668359552:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.574103:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784691.574534:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014080 00000100:00100000:1.0:1787784691.574544:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014080 00000100:00100000:1.0:1787784691.574667:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 464 00000100:00100000:1.0:1787784691.574677:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658014080:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784691.574690:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784691.574707:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09bc expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.574733:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.574743:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.574753:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640c00, rc 0) 00000100:00100000:1.0:1787784691.574814:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658014080:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 139us (721us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.574833:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 464 00000100:00100000:3.0:1787784691.575228:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668359552:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 1897us (2378us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784691.575236:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:1.0:1787784691.576743:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359552:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.576806:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359680:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.576868:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.576910:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784691.576989:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359680 00000100:00100000:0.0:1787784691.577004:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359680 00000100:00100000:0.0:1787784691.577073:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:0.0:1787784691.577086:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668359680:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.577449:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668359680:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 366us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784691.577459:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1787784691.577674:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359680:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.580018:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000040:00080000:3.0:1787784691.580265:0:17812:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000100:00100000:2.0:1787784691.580311:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014208 00000100:00100000:2.0:1787784691.580325:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014208 00000100:00100000:2.0:1787784691.580408:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:2.0:1787784691.580420:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658014208:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000020:01000000:1.0:1787784691.580552:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:1.0:1787784691.580669:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:1.0:1787784691.580681:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.580696:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784691.580708:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787784691.580711:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9895054dd800 from 49 10000000:01000000:1.0:1787784691.580715:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787784691.580756:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.580767:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.580779:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784691.580805:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359808:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.580887:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.580930:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.580981:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658014208:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 559us (971us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.580992:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:2.0:1787784691.581025:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359808 00000100:00100000:2.0:1787784691.581038:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359808 00000100:00100000:2.0:1787784691.581060:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:2.0:1787784691.581067:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668359808:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784691.581077:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784691.581096:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d4d expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.581124:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.581136:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.581182:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea600, rc 0) 00000100:00100000:2.0:1787784691.581229:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668359808:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 162us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.581236:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:1.0:1787784691.582095:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359808:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784691.582118:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784691.582130:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784691.582216:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359936:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.582982:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.583019:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.585058:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668359936 00000100:00100000:2.0:1787784691.585078:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668359936 00000100:00100000:2.0:1787784691.585177:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1787784691.585189:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668359936:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.585588:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668359936:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 400us (2606us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.585597:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:1.0:1787784691.585720:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668359936:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.585796:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360064:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.585820:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784691.585876:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.585916:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014336 00000100:00100000:1.0:1787784691.585923:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014336 00000100:00100000:1.0:1787784691.585961:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360064 00000100:00100000:1.0:1787784691.585964:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360064 00000100:00100000:1.0:1787784691.585988:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00100000:1.0:1787784691.585998:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658014336:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.586009:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1787784691.586021:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668360064:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.586272:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668360064:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 254us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.586286:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:1.0:1787784691.591644:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658014336:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 5568us (5748us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.591660:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00100000:1.0:1787784691.591727:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.591754:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360064:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784691.595515:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17813:1874627668360192:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784691.595621:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784691.595668:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950631e180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.595857:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360192 00000100:00100000:2.0:1787784691.595865:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360192 00000100:00100000:2.0:1787784691.595914:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1787784691.595923:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:17813:x1874627668360192:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.596224:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:17813:x1874627668360192:12345-0@lo:502:llog_process_th.0 Request processed in 302us (603us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.596233:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:1.0:1787784691.596346:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17813:1874627668360192:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784691.596401:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17813:1874627668360320:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784691.596497:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784691.596527:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.596684:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360320 00000100:00100000:2.0:1787784691.596689:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360320 00000100:00100000:2.0:1787784691.596723:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:2.0:1787784691.596731:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17813:x1874627668360320:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.596847:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17813:x1874627668360320:12345-0@lo:502:llog_process_th.0 Request processed in 116us (348us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.596854:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:1.0:1787784691.598661:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17813:1874627668360320:0@lo:502:llog_process_th.0 00000040:00080000:1.0:1787784691.598707:0:17813:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000020:01000000:2.0:1787784691.598940:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 49-51 (rc=0) 10000000:01000000:2.0:1787784691.598961:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1787784691.598971:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.598983:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784691.598993:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:2.0:1787784691.598996:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 43 10000000:01000000:2.0:1787784691.598998:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:2.0:1787784691.599031:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.599048:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.599057:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784691.599077:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360448:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.599126:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.599206:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1600 going to sleep for 11 seconds 00000100:00100000:0.0:1787784691.599279:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360448 00000100:00100000:0.0:1787784691.599286:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360448 00000100:00100000:0.0:1787784691.599337:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1787784691.599346:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668360448:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784691.599358:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784691.599371:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d5b expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.599394:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.599405:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.599424:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbca00, rc 0) 00000100:00100000:0.0:1787784691.599459:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668360448:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 115us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784691.599466:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:2.0:1787784691.599726:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360448:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784691.599740:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.599750:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784691.599801:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360576:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.599830:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.599858:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.600017:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360576 00000100:00100000:1.0:1787784691.600022:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360576 00000100:00100000:1.0:1787784691.600073:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1787784691.600227:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668360576:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.600597:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668360576:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 516us (765us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.600605:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:2.0:1787784691.600724:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360576:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.600757:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360704:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.600798:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.600821:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360704 00000100:00100000:2.0:1787784691.600825:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360704 00000100:00100000:2.0:1787784691.600843:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:2.0:1787784691.600850:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668360704:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.600990:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668360704:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 139us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.600997:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:2.0:1787784691.601034:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.601045:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668360704:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.601228:0:17814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17814:1874627668360832:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.601269:0:17814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.601285:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360832 00000100:00100000:2.0:1787784691.601289:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360832 00000100:00100000:2.0:1787784691.601306:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1787784691.601311:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:17814:x1874627668360832:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.601456:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:17814:x1874627668360832:12345-0@lo:502:llog_process_th.0 Request processed in 144us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.601464:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1787784691.601488:0:17814:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1300 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.601501:0:17814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17814:1874627668360832:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1787784691.601522:0:17814:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787784691.601532:0:17814:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787784691.601535:0:17814:0:(obd_config.c:1563:class_process_config()) marker 33 (0x1) lustre-MDT0000-m rem lustre.test_11c.lustre-OST0001_UUID 00000020:00000080:2.0:1787784691.601540:0:17814:0:(obd_config.c:1472:class_process_config()) processing cmd: ce022 00020000:01000000:2.0:1787784691.601554:0:17814:0:(lod_pool.c:649:lod_pool_remove()) lustre-OST0001_UUID removed from test_11c 00000100:00100000:2.0:1787784691.611667:0:17814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17814:1874627668360960:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.611755:0:17814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.611797:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668360960 00000100:00100000:2.0:1787784691.611804:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668360960 00000100:00100000:2.0:1787784691.611858:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:2.0:1787784691.611866:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:17814:x1874627668360960:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784691.612099:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+18:17814:x1874627668360960:12345-0@lo:502:llog_process_th.0 Request processed in 233us (344us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.612107:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:2.0:1787784691.612188:0:17814:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1300 going to sleep for 11 seconds 00000100:00100000:2.0:1787784691.612211:0:17814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17814:1874627668360960:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1787784691.612236:0:17814:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787784691.612252:0:17814:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787784691.612255:0:17814:0:(obd_config.c:1563:class_process_config()) marker 33 (0x2) lustre-MDT0000-m rem lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:2.0:1787784691.612259:0:17814:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 46 count 46 00000020:01000000:2.0:1787784691.612310:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 43-45 (rc=0) 10000000:01000000:2.0:1787784691.612326:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:2.0:1787784691.612336:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.612346:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784691.612356:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787784691.612358:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff989508964d28 from 1 10000000:01000000:2.0:1787784691.612361:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1787784691.612386:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.612394:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.612402:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784691.612434:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668361088:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.612462:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.612475:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1300 going to sleep for 11 seconds 00000100:00100000:0.0:1787784691.612743:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361088 00000100:00100000:0.0:1787784691.612748:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361088 00000100:00100000:0.0:1787784691.612782:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:0.0:1787784691.612789:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668361088:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784691.612798:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784691.612812:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d69 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.612834:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 19 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.612846:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 19 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784691.612866:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc400, rc 0) 00000100:00100000:0.0:1787784691.612898:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668361088:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 110us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784691.612905:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000020:01000004:0.0:1787784691.612969:0:17815:0:(tgt_mount.c:153:server_get_mount()) get mount ffff9895054dd800 from lustre-MDT0000, refs=4 00000020:01000004:0.0:1787784691.612972:0:17815:0:(obd_mount.c:710:lustre_put_lsi()) put ffff9895054dd800 4 00000100:00100000:2.0:1787784691.613638:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668361088:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784691.613656:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.613666:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784691.613714:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668361216:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.613756:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784691.613823:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1300 going to sleep for 11 seconds 00000100:00100000:1.0:1787784691.614322:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784691.621828:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361216 00000100:00100000:0.0:1787784691.621844:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361216 00000100:00100000:0.0:1787784691.621947:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014464 00000100:00100000:0.0:1787784691.621951:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014464 00000100:00100000:2.0:1787784691.624005:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:2.0:1787784691.624026:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668361216:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784691.624349:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668361216:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 331us (10591us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787784691.624358:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:2.0:1787784691.624421:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668361216:0@lo:501:ll_cfg_requeue.0 10000000:01000000:2.0:1787784691.624459:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:2.0:1787784691.624468:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.624479:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784691.624519:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 480 00000100:00100000:2.0:1787784691.624527:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23021:x1874627658014464:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784691.625058:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23021:x1874627658014464:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 530us (10736us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.625066:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 480 00000100:00100000:0.0:1787784691.632740:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784691.633139:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014592 00000100:00100000:1.0:1787784691.633163:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014592 00000100:00100000:1.0:1787784691.633215:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 481 00000100:00100000:1.0:1787784691.633225:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23021:x1874627658014592:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:1.0:1787784691.633480:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23021:x1874627658014592:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 255us (749us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784691.633488:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 481 00010000:00010000:2.0:1787784691.640505:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784691.642797:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784691.643329:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014720 00000100:00100000:2.0:1787784691.643338:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014720 00000100:00100000:2.0:1787784691.643422:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 482 00000100:00100000:2.0:1787784691.643432:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658014720:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784691.643449:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784691.643466:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09c3 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.643495:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.643506:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784691.643515:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eaa00, rc 0) 00000100:00100000:2.0:1787784691.643576:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658014720:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 145us (788us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784691.643583:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 482 00000100:00100000:3.0:1787784691.648530:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784691.652890:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014848 00000100:00100000:1.0:1787784691.652915:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014848 00000100:00100000:1.0:1787784691.652978:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 483 00000100:00100000:1.0:1787784691.652988:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658014848:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784691.653428:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658014848:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 440us (4907us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784691.653438:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 483 00000040:00080000:1.0:1787784692.146020:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784692.146030:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784692.146715:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784692.146725:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784692.209413:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668361728:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209479:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784692.209532:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784692.209538:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668361600:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209563:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784692.209596:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361728 00000100:00100000:3.0:1787784692.209603:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361728 00000100:00100000:3.0:1787784692.209649:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:3.0:1787784692.209657:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668361728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209704:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668361728:12345-0@lo:400:kworker.0 Request processed in 47us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784692.209710:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:3.0:1787784692.209730:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361600 00000100:00100000:3.0:1787784692.209733:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361600 00000100:00100000:3.0:1787784692.209758:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:3.0:1787784692.209763:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668361600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209790:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668361600:12345-0@lo:400:kworker.0 Request processed in 27us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784692.209810:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00080000:3.0:1787784692.209836:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784692.209840:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784692.209874:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668361600:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209884:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668361728:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209916:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668361472:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209940:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784692.209956:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668361984:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.209977:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784692.209994:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787784692.209999:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668361344:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210024:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784692.210045:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668361856:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210064:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784692.210085:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361984 00000100:00100000:3.0:1787784692.210088:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361984 00000100:00100000:3.0:1787784692.210114:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1787784692.210119:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668361984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210177:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668361984:12345-0@lo:400:kworker.0 Request processed in 57us (199us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784692.210184:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1787784692.210200:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361472 00000100:00100000:3.0:1787784692.210203:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361472 00000100:00100000:3.0:1787784692.210222:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361856 00000100:00100000:3.0:1787784692.210225:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361856 00000100:00100000:3.0:1787784692.210253:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1787784692.210260:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668361472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210298:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668361472:12345-0@lo:400:kworker.0 Request processed in 39us (357us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784692.210305:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1787784692.210315:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:3.0:1787784692.210320:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668361856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210344:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668361856:12345-0@lo:400:kworker.0 Request processed in 25us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784692.210349:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00080000:3.0:1787784692.210367:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784692.210370:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784692.210388:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668361856:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210398:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668361472:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210406:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668361984:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210442:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668361344 00000100:00100000:3.0:1787784692.210445:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668361344 00000100:00100000:3.0:1787784692.210467:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1787784692.210474:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668361344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.210508:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668361344:12345-0@lo:400:kworker.0 Request processed in 34us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784692.210513:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1787784692.210536:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668361344:0@lo:400:kworker.0 00000100:00100000:3.0:1787784692.225299:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668362112:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784692.225365:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784692.228404:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668362112 00000100:00100000:1.0:1787784692.228418:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668362112 00000100:00100000:1.0:1787784692.228476:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:1.0:1787784692.228487:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668362112:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784692.228525:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784692.228608:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668362112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 123us (3242us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784692.228616:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:1.0:1787784692.228670:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668362112:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1787784692.659462:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784693.684781:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1787784694.158516:0:17915:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007ffe27aff210 20000000:00000080:2.0:1787784694.158548:0:17915:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff989402013c00 uarg=0000000000000000 00000040:00100000:2.0:1787784694.159499:0:17915:0:(llog.c:1072:llog_reverse_process()) index 51 : idx 48 20000000:01000000:2.0:1787784694.159511:0:17915:0:(mgs_llog.c:963:mgs_search_pool_cb()) Matched pool rec 50 cmd:0xce022 lustre.test_11c lustre-OST0001_UUID 20000000:01000000:2.0:1787784694.159528:0:17915:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-client/lustre-clilov/new lustre.test_11c fl=4 20000000:01000000:0.0:1787784694.174595:0:17916:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 42 marker 28 1->4: lustre-clilov new lustre.test_11c 20000000:01000000:0.0:1787784694.174751:0:17916:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 44 marker 28 2->4: lustre-clilov new lustre.test_11c 00000040:00080000:0.0:1787784694.174833:0:17916:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000040:00080000:3.0:1787784694.176286:0:17915:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].52, 224 off17416 20000000:01000000:3.0:1787784694.176311:0:17915:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce023 lustre test_11c (null) (null) 00000040:00080000:3.0:1787784694.176493:0:17915:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].53, 112 off17528 00000040:00080000:3.0:1787784694.176602:0:17915:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].54, 224 off17752 20000000:01000000:3.0:1787784694.176622:0:17915:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-MDT0000/lustre-MDT0000-mdtlov/new lustre.test_11c fl=4 20000000:01000000:0.0:1787784694.181166:0:17917:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 37 marker 29 1->4: lustre-MDT0000-mdtlov new lustre.test_11c 20000000:01000000:0.0:1787784694.181385:0:17917:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 39 marker 29 2->4: lustre-MDT0000-mdtlov new lustre.test_11c 00000040:00080000:0.0:1787784694.181464:0:17917:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 46 count 46 00000040:00080000:3.0:1787784694.182437:0:17915:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].47, 224 off16608 20000000:01000000:3.0:1787784694.182461:0:17915:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce023 lustre test_11c (null) (null) 00000040:00080000:3.0:1787784694.182498:0:17915:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].48, 120 off16728 00000040:00080000:3.0:1787784694.182567:0:17915:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].49, 224 off16952 10000000:01000000:3.0:1787784694.182586:0:17915:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:3.0:1787784694.182612:0:17915:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182635:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182644:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182655:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182663:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182670:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182678:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182684:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182699:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182706:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182713:0:17915:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182722:0:17915:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182787:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182810:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 12 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.182829:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f09a40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362240:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784694.182981:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.182992:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 18 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183003:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362368:0@lo:104: 00000100:00100000:3.0:1787784694.183066:0:17915:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784694.183111:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.183128:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 18 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183141:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362496:0@lo:104: 00000100:00100000:3.0:1787784694.183214:0:17915:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784694.183258:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.183266:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 18 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183275:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0a940 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362624:0@lo:104: 00000100:00100000:3.0:1787784694.183298:0:17915:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784694.183329:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.183337:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183346:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362752:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784694.183443:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.183451:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 18 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183459:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362880:0@lo:104: 00000100:00100000:3.0:1787784694.183486:0:17915:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784694.183501:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668362368 00000100:00100000:1.0:1787784694.183517:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668362368 00010000:00010000:3.0:1787784694.183522:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.183534:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 18 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183543:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363008:0@lo:104: 00000100:00100000:1.0:1787784694.183570:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668362496 00000100:00100000:3.0:1787784694.183579:0:17915:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784694.183580:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668362496 00000100:00100000:1.0:1787784694.183607:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668362624 00010000:00010000:3.0:1787784694.183612:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.183614:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668362624 00010000:00010000:3.0:1787784694.183630:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 12 pid: 5804 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.183636:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668362880 00000100:00100000:1.0:1787784694.183643:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668362880 00000100:00100000:3.0:1787784694.183644:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073a940 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363136:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784694.183668:0:5793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668363008 00000100:00100000:1.0:1787784694.183675:0:5793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668363008 00010000:00010000:3.0:1787784694.183697:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.183703:0:5793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00010000:00010000:3.0:1787784694.183712:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 18 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183726:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363264:0@lo:104: 00000100:00100000:1.0:1787784694.183737:0:5793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+7:17915:x1874627668362368:12345-0@lo:104: 00000100:00100000:3.0:1787784694.183756:0:17915:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787784694.183759:0:5793:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.183778:0:17915:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.183790:0:17915:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 18 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.183799:0:17915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363392:0@lo:104: 00000100:00100000:0.0:1787784694.183993:0:5794:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668363264 00000100:00100000:0.0:1787784694.184001:0:5794:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668363264 00000100:00100000:0.0:1787784694.184047:0:5794:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:0.0:1787784694.184056:0:5794:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+8:17915:x1874627668362496:12345-0@lo:104: 00010000:00010000:0.0:1787784694.184069:0:5794:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.185514:0:17915:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784694.185589:0:17915:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1000 going to sleep for 11 seconds 00000100:00100000:3.0:1787784694.185640:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362368:0@lo:104: 00010000:00010000:1.0:1787784694.185682:0:5793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787784694.185698:0:5793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a5a93c05-6249-4607-a205-909a326a8fd1+8:17915:x1874627668362368:12345-0@lo:104: Request processed in 1963us (2628us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.185710:0:5793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:3.0:1787784694.185724:0:8493:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787784694.185744:0:8493:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668363392 00000100:00100000:3.0:1787784694.185750:0:8493:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668363392 00000100:00100000:3.0:1787784694.185848:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:3.0:1787784694.185857:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+9:17915:x1874627668362624:12345-0@lo:104: 00010000:00010000:3.0:1787784694.185873:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.186067:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:3.0:1787784694.186074:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+9:17915:x1874627668362624:12345-0@lo:104: Request processed in 218us (2775us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784694.186081:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000100:00100000:3.0:1787784694.186103:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:3.0:1787784694.186110:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+8:17915:x1874627668362880:12345-0@lo:104: 00010000:00010000:3.0:1787784694.186118:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.186175:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:3.0:1787784694.186180:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+8:17915:x1874627668362880:12345-0@lo:104: Request processed in 71us (2697us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784694.186186:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:3.0:1787784694.186199:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:3.0:1787784694.186205:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+7:17915:x1874627668363008:12345-0@lo:104: 00010000:00010000:3.0:1787784694.186213:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188265:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787784694.188279:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188291:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eb200 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:1.0:1787784694.188296:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188306:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787784694.188314:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784694.188320:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787784694.188374:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188381:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787784694.188404:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188431:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff9894032eb200/0xb0ce63137acc4d69 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d70 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188445:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787784694.188449:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188454:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640a00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:1.0:1787784694.188457:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188464:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787784694.188470:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784694.188475:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787784694.188492:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188498:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787784694.188513:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188520:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4d4d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d54 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188529:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784694.188539:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188545:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbdc00 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:1.0:1787784694.188548:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188554:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787784694.188559:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784694.188564:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787784694.188578:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188584:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787784694.188590:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188596:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbdc00/0xb0ce63137acc4d38 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d3f expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.188604:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784694.188606:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784694.188608:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784694.188624:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668363520:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784694.188685:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784694.188745:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668363648:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784694.188780:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784694.188798:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784694.188804:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668363776:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784694.188837:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784694.188863:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668363520 00000100:00100000:1.0:1787784694.188869:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668363520 00010000:00010000:1.0:1787784694.188898:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 19 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.188921:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668363648 00000100:00100000:1.0:1787784694.188924:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668363648 00010000:00010000:1.0:1787784694.188936:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 20 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.188947:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668363776 00000100:00100000:1.0:1787784694.188949:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668363776 00010000:00010000:1.0:1787784694.188958:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 21 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.189833:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:1.0:1787784694.189849:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057efc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a5a93c05-6249-4607-a205-909a326a8fd1+21:3279:x1874627668363520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.189874:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.190116:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 21 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.190129:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 21 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.190169:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbc400/0xb0ce63137acc4d70 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4d69 expref: 21 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.190201:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.190271:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057efc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a5a93c05-6249-4607-a205-909a326a8fd1+20:3279:x1874627668363520:12345-0@lo:103:ldlm_bl.0 Request processed in 427us (1584us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.190278:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:1.0:1787784694.190326:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:1.0:1787784694.190332:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668363648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.190340:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.190344:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 19 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.190352:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 19 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.190361:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.190393:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668363648:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (1614us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.190399:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00010000:00010000:3.0:1787784694.190458:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787784694.190473:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+6:17915:x1874627668363008:12345-0@lo:104: Request processed in 4264us (6895us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784694.190485:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000100:00100000:3.0:1787784694.190520:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:3.0:1787784694.190531:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+5:17915:x1874627668363264:12345-0@lo:104: 00010000:00010000:3.0:1787784694.190548:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.190590:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:3.0:1787784694.190595:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+5:17915:x1874627668363264:12345-0@lo:104: Request processed in 68us (6842us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784694.190602:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:3.0:1787784694.190629:0:8493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 55 00000100:00100000:3.0:1787784694.190636:0:8493:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011b4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17915:x1874627668363392:12345-0@lo:104: 00010000:00010000:3.0:1787784694.190645:0:8493:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.190661:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:1.0:1787784694.190679:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a5a93c05-6249-4607-a205-909a326a8fd1+18:3276:x1874627668363776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784694.190687:0:8493:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00010000:00010000:1.0:1787784694.190690:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787784694.190698:0:8493:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011b4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a5a93c05-6249-4607-a205-909a326a8fd1+4:17915:x1874627668363392:12345-0@lo:104: Request processed in 59us (5192us total) trans 0 rc 0/0 00010000:00010000:1.0:1787784694.190701:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 18 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.190716:0:8493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 55 00010000:00010000:1.0:1787784694.190731:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 18 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.190745:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.190786:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a5a93c05-6249-4607-a205-909a326a8fd1+18:3276:x1874627668363776:12345-0@lo:103:ldlm_bl.0 Request processed in 115us (1947us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.190792:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:1.0:1787784694.190849:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668363776:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784694.190873:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668363648:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784694.190897:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787784694.190906:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.190928:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668363520:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784694.190947:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784694.190953:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.190960:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787784694.190964:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784694.190974:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784694.190981:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784694.191017:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191024:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787784694.191053:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191061:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989401dbd000/0xb0ce63137acc4d2a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d31 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191077:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784694.191080:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191087:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784694.191091:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191110:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784694.191120:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.191121:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668363904:0@lo:103:ldlm_bl.0 10000000:01000000:3.0:1787784694.191136:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784694.191188:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191195:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787784694.191204:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784694.191206:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191216:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff989403640400/0xb0ce63137acc4d07 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d0e expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.191257:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784694.191265:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668364032:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784694.191286:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784694.191312:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668363904 00000100:00100000:1.0:1787784694.191317:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668363904 00010000:00010000:1.0:1787784694.191338:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 18 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.191357:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364032 00000100:00100000:1.0:1787784694.191360:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364032 00010000:00010000:1.0:1787784694.191370:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 19 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.191385:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:1.0:1787784694.191390:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668363904:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.191398:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.191402:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 19 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191488:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 19 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191501:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.191559:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668363904:12345-0@lo:103:ldlm_bl.0 Request processed in 164us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.191566:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00010000:00010000:0.0:1787784694.191571:0:5794:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787784694.191585:0:5794:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a5a93c05-6249-4607-a205-909a326a8fd1+3:17915:x1874627668362496:12345-0@lo:104: Request processed in 7524us (8364us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.191594:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:1.0:1787784694.191611:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+18:3278:x1874627668364032:12345-0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787784694.191627:0:5794:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00010000:00010000:1.0:1787784694.191630:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.191638:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 18 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191647:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 18 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191661:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.191694:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+18:3278:x1874627668364032:12345-0@lo:103:ldlm_bl.0 Request processed in 92us (407us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.191701:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:1.0:1787784694.191740:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668364032:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787784694.191747:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668363904:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784694.191836:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784694.191838:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784694.191844:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191848:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191862:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032ea000 already unused, calling callback (ffffffffc1da1c20) 10000000:00010000:3.0:1787784694.191870:0:5799:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191874:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894032eba00 already unused, calling callback (ffffffffc1da1c20) 00010000:00010000:3.0:1787784694.191882:0:5799:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787784694.191889:0:5798:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787784694.191893:0:5799:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191903:0:5798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784694.191913:0:5799:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:00010000:1.0:1787784694.191930:0:5798:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784694.191938:0:5798:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787784694.191943:0:5799:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191951:0:5799:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787784694.191969:0:5798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.191974:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191980:0:5798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787784694.191990:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff9894032ea000/0xb0ce63137acc4d5b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d62 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.191997:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.192004:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784694.192008:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784694.192008:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.143@tcp lock: ffff9894032eba00/0xb0ce63137acc4d15 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xb0ce63137acc4d1c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.192013:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784694.192038:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784694.192042:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784694.192045:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784694.192060:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668364160:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787784694.192067:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362496:0@lo:104: 00010000:00010000:3.0:1787784694.192088:0:17915:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4d54 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4d4d expref: 17 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.192107:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0a940 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362624:0@lo:104: 00000100:00100000:1.0:1787784694.192113:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787784694.192131:0:17915:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff98953f5b3e00/0xb0ce63137acc4d3f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4d38 expref: 16 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.192196:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362880:0@lo:104: 00000100:00100000:1.0:1787784694.192197:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784694.192215:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668364288:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784694.192221:0:17915:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4d31 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4d2a expref: 15 pid: 5805 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.192587:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363008:0@lo:104: 00000100:00100000:3.0:1787784694.192604:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363264:0@lo:104: 00010000:00010000:3.0:1787784694.192623:0:17915:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032ea800/0xb0ce63137acc4d0e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4d07 expref: 14 pid: 5804 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.192631:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363392:0@lo:104: 00000100:00100000:3.0:1787784694.192964:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784694.192990:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784694.193044:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784694.193077:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f09a40 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362240:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784694.193087:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668362752:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784694.193689:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784694.193745:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364160 00000100:00100000:1.0:1787784694.193753:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364160 00010000:00010000:1.0:1787784694.193796:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 14 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.193819:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658014976 00000100:00100000:1.0:1787784694.193822:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658014976 00010000:00010000:1.0:1787784694.193840:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 13 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.193851:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658015104 00000100:00100000:1.0:1787784694.193853:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658015104 00010000:00010000:1.0:1787784694.193870:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 14 pid: 5804 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.194252:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658015232 00000100:00100000:1.0:1787784694.194257:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658015232 00010000:00010000:1.0:1787784694.194273:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 15 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.194283:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364288 00000100:00100000:1.0:1787784694.194285:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364288 00010000:00010000:1.0:1787784694.194360:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 15 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784694.194375:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:1.0:1787784694.194382:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+15:3276:x1874627668364160:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.194394:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.194399:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 15 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.194409:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 15 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.194504:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbca00/0xb0ce63137acc4d62 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4d5b expref: 15 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.194518:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.194647:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+14:3276:x1874627668364160:12345-0@lo:103:ldlm_bl.0 Request processed in 265us (2545us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.194655:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:1.0:1787784694.194674:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:1.0:1787784694.194681:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+15:2357:x1874627658014976:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.194691:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.194696:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 15 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.194707:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 15 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.194718:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d77 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09c3 expref: 15 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.194727:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.194762:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:2357:x1874627658014976:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 81us (1802us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.194769:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:1.0:1787784694.194783:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 241 00000100:00100000:1.0:1787784694.194789:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ed050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658015104:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.194797:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.194807:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 13 pid: 5804 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.194815:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 13 pid: 5804 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.196596:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.196654:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ed050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658015104:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1858us (3656us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.196664:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 241 00000100:00100000:1.0:1787784694.196725:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 242 00000100:00100000:1.0:1787784694.196736:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:2356:x1874627658015232:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.196751:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.196757:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.196770:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.196780:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989403640c00/0xb0ce63137acc4d46 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09bc expref: 12 pid: 5805 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.196794:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.196815:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:2356:x1874627658015232:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 81us (3771us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.196821:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 242 00000100:00100000:1.0:1787784694.196836:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:1.0:1787784694.196843:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+13:3276:x1874627668364288:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787784694.196867:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784694.196872:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 13 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.197447:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668364160:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784694.197519:0:17915:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989401dbc600/0xb0ce63137acc4d23 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09b5 expref: 10 pid: 5804 timeout: 585 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784694.197536:0:17915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073a940 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17915:1874627668363136:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784694.198342:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 13 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784694.199215:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784694.199369:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784694.200383:0:17915:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201042:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989403640600/0xb0ce63137acc4d1c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0xb0ce63137acc4d15 expref: 13 pid: 5803 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201066:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201075:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201098:0:5797:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201106:0:5797:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201112:0:5797:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201120:0:5797:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201126:0:5797:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784694.201129:0:5797:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201177:0:5797:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201199:0:5797:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784694.201214:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784694.201313:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a5a93c05-6249-4607-a205-909a326a8fd1+12:3276:x1874627668364288:12345-0@lo:103:ldlm_bl.0 Request processed in 4468us (7627us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784694.201324:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00010000:00010000:3.0:1787784694.201388:0:17915:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.201404:0:17915:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784694.201411:0:17915:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff98953f5b3600/0xb0ce63137acc4d7e lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17915 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784694.203337:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668364288:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787784694.706042:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787784695.281430:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787784695.281441:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989518f73800 from 52 10000000:01000000:2.0:1787784695.281448:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787784695.281492:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea400/0xb0ce63137acc4d85 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.281506:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032ea400/0xb0ce63137acc4d85 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.281519:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032ea400/0xb0ce63137acc4d85 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784695.281547:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668364544:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.281556:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784695.281598:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668364416:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784695.282570:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.282914:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364416 00000100:00100000:1.0:1787784695.282923:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364416 00000100:00100000:1.0:1787784695.283041:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:1.0:1787784695.283051:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668364416:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784695.283091:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784695.283139:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668364416:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (572us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.283171:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:0.0:1787784695.283357:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668364416:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787784695.283723:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784695.284846:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.285012:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066770c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784695.287853:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364544 00000100:00100000:0.0:1787784695.287874:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364544 00000100:00100000:0.0:1787784695.287975:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:0.0:1787784695.287988:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+12:5808:x1874627668364544:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784695.288010:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784695.288037:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4d8c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d85 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.288073:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4d8c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d85 expref: 13 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.288087:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbde00/0xb0ce63137acc4d8c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d85 expref: 13 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.288098:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbde00, rc 0) 00000100:00100000:0.0:1787784695.289060:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668364544:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 1068us (4216us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.289072:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:2.0:1787784695.296410:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668364544:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784695.296452:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032ea400/0xb0ce63137acc4d85 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.296467:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032ea400/0xb0ce63137acc4d85 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784695.296579:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668364672:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.297365:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.297541:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066770c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784695.299526:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364672 00000100:00100000:1.0:1787784695.299549:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364672 00000100:00100000:1.0:1787784695.299646:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:1.0:1787784695.300559:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668364672:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.304520:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668364672:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 3963us (7143us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.305324:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:2.0:1787784695.307000:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668364672:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.307094:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668364800:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.307320:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.307449:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066770c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784695.308796:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364800 00000100:00100000:0.0:1787784695.308812:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364800 00000100:00100000:0.0:1787784695.308977:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:0.0:1787784695.308989:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668364800:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.309314:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668364800:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 327us (1994us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.309322:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:2.0:1787784695.310063:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668364800:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.315682:0:17919:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17919:1874627668364928:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.315810:0:17919:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.322897:0:17919:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784695.326451:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668364928 00000100:00100000:3.0:1787784695.326472:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668364928 00000100:00100000:3.0:1787784695.326546:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 488 00000100:00100000:3.0:1787784695.326558:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:17919:x1874627668364928:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784695.327121:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+13:17919:x1874627668364928:12345-0@lo:502:llog_process_th.0 Request processed in 561us (11308us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784695.327132:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 488 00000100:00100000:2.0:1787784695.327354:0:17919:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17919:1874627668364928:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.327452:0:17919:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17919:1874627668365056:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.327513:0:17919:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.327562:0:17919:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033becc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784695.327644:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365056 00000100:00100000:0.0:1787784695.327829:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365056 00000100:00100000:0.0:1787784695.327871:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:0.0:1787784695.327881:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:17919:x1874627668365056:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784695.328075:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:17919:x1874627668365056:12345-0@lo:502:llog_process_th.0 Request processed in 195us (561us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.328083:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:2.0:1787784695.328177:0:17919:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17919:1874627668365056:0@lo:502:llog_process_th.0 00000040:00080000:2.0:1787784695.328203:0:17919:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 55 count 54 00000020:01000000:0.0:1787784695.328729:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-54 (rc=0) 10000000:01000000:0.0:1787784695.328757:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787784695.328772:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea400/0xb0ce63137acc4d85 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.328786:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032ea400/0xb0ce63137acc4d85 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d8c expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787784695.328799:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:0.0:1787784695.328803:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:0.0:1787784695.328807:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:0.0:1787784695.328847:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4d93 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.328857:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4d93 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.328885:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4d93 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784695.328908:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365184:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.328961:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784695.329006:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.329058:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365184 00000100:00100000:2.0:1787784695.329064:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365184 00000100:00100000:2.0:1787784695.329101:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:2.0:1787784695.329109:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+13:5808:x1874627668365184:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784695.329121:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784695.329135:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d9a lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4d93 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.329208:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d9a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4d93 expref: 14 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.329243:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eaa00/0xb0ce63137acc4d9a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4d93 expref: 14 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.329253:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eaa00, rc 0) 00000100:00100000:2.0:1787784695.329289:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668365184:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 181us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.329296:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:0.0:1787784695.331492:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365184:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784695.331532:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4d93 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.331546:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4d93 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4d9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784695.331645:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365312:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.331772:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784695.331832:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.332105:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365312 00000100:00100000:2.0:1787784695.332113:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365312 00000100:00100000:2.0:1787784695.332189:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:2.0:1787784695.332200:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668365312:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.332569:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668365312:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 370us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.332579:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:0.0:1787784695.332784:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365312:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.332830:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365440:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.332880:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784695.332915:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.333767:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365440 00000100:00100000:2.0:1787784695.333775:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365440 00000100:00100000:2.0:1787784695.333823:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1787784695.333832:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668365440:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.334034:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668365440:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 202us (1153us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.334042:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:0.0:1787784695.334535:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365440:0@lo:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787784695.342529:0:17920:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:1.0:1787784695.342802:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:1.0:1787784695.342828:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:1.0:1787784695.342860:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4d93 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.342873:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbc600/0xb0ce63137acc4d93 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4d9a expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784695.342886:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787784695.342889:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff989538a62800 from 52 10000000:01000000:1.0:1787784695.342891:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787784695.342937:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640600/0xb0ce63137acc4da1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.342945:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989403640600/0xb0ce63137acc4da1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.342957:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989403640600/0xb0ce63137acc4da1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784695.342991:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365568:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.343081:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.343135:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.343251:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365568 00000100:00100000:2.0:1787784695.343260:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365568 00000100:00100000:2.0:1787784695.343316:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1787784695.343327:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+14:5808:x1874627668365568:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784695.343344:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784695.343367:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4da8 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4da1 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.343395:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4da8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4da1 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.343408:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eba00/0xb0ce63137acc4da8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4da1 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.343418:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eba00, rc 0) 00000100:00100000:2.0:1787784695.343463:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668365568:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 138us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.343470:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:1.0:1787784695.343627:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365568:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784695.343643:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989403640600/0xb0ce63137acc4da1 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4da8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.343653:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989403640600/0xb0ce63137acc4da1 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4da8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784695.343713:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365696:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.343753:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.343781:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.343823:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365696 00000100:00100000:2.0:1787784695.343828:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365696 00000100:00100000:2.0:1787784695.343863:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1787784695.343871:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668365696:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.344213:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668365696:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 341us (457us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.344229:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:1.0:1787784695.344351:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365696:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.344384:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365824:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.344429:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.344456:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784695.344525:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365824 00000100:00100000:3.0:1787784695.344530:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365824 00000100:00100000:3.0:1787784695.344563:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:3.0:1787784695.344572:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668365824:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.344748:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668365824:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 177us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784695.344756:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:3.0:1787784695.344794:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668365824:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.345334:0:17921:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17921:1874627668365952:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784695.345377:0:17921:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784695.345408:0:17921:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.345449:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668365952 00000100:00100000:2.0:1787784695.345454:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668365952 00000100:00100000:2.0:1787784695.345479:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 496 00000100:00100000:2.0:1787784695.345489:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:17921:x1874627668365952:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.345708:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+15:17921:x1874627668365952:12345-0@lo:502:llog_process_th.0 Request processed in 221us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.345716:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 496 00000100:00100000:3.0:1787784695.345828:0:17921:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17921:1874627668365952:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784695.345872:0:17921:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17921:1874627668366080:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1787784695.345904:0:17921:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784695.345926:0:17921:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.345968:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366080 00000100:00100000:2.0:1787784695.345979:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366080 00000100:00100000:2.0:1787784695.346008:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:2.0:1787784695.346013:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17921:x1874627668366080:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.346136:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+15:17921:x1874627668366080:12345-0@lo:502:llog_process_th.0 Request processed in 122us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.346143:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:1.0:1787784695.346291:0:17921:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17921:1874627668366080:0@lo:502:llog_process_th.0 00000040:00080000:1.0:1787784695.346313:0:17921:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 55 count 54 00000020:01000000:2.0:1787784695.347506:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-54 (rc=0) 10000000:01000000:2.0:1787784695.347542:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1787784695.347556:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640600/0xb0ce63137acc4da1 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4da8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.347570:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989403640600/0xb0ce63137acc4da1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4da8 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787784695.347583:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:2.0:1787784695.347586:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:2.0:1787784695.347590:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:2.0:1787784695.347627:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea800/0xb0ce63137acc4daf lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.347635:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff9894032ea800/0xb0ce63137acc4daf lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.347646:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff9894032ea800/0xb0ce63137acc4daf lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784695.347671:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366208:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.347733:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.347771:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787784695.347844:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366208 00000100:00100000:1.0:1787784695.347852:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366208 00000100:00100000:1.0:1787784695.347898:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:1.0:1787784695.347909:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+15:5808:x1874627668366208:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784695.347921:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784695.347937:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989403640c00/0xb0ce63137acc4db6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4daf expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.347964:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989403640c00/0xb0ce63137acc4db6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4daf expref: 16 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.347975:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989403640c00/0xb0ce63137acc4db6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4daf expref: 16 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.347985:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640c00, rc 0) 00000100:00100000:1.0:1787784695.348040:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668366208:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 133us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.348046:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00100000:3.0:1787784695.348684:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366208:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784695.348705:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff9894032ea800/0xb0ce63137acc4daf lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4db6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.348727:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff9894032ea800/0xb0ce63137acc4daf lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4db6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784695.348798:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366336:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.348841:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784695.348869:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1540 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.348954:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366336 00000100:00100000:2.0:1787784695.348960:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366336 00000100:00100000:2.0:1787784695.349005:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:2.0:1787784695.349014:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668366336:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.352723:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668366336:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 3695us (3868us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.352741:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:3.0:1787784695.353004:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366336:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.353068:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366464:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.353123:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784695.353178:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1540 going to sleep for 11 seconds 00000100:00100000:0.0:1787784695.353258:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366464 00000100:00100000:0.0:1787784695.353267:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366464 00000100:00100000:0.0:1787784695.353318:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 500 00000100:00100000:0.0:1787784695.353329:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668366464:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.353542:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668366464:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 215us (418us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.353550:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 500 00000100:00100000:2.0:1787784695.353736:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366464:0@lo:503:ll_cfg_requeue.0 00000040:00080000:2.0:1787784695.354659:0:17922:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000020:01000000:3.0:1787784695.358351:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:3.0:1787784695.358406:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:3.0:1787784695.358422:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff9894032ea800/0xb0ce63137acc4daf lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4db6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.358439:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff9894032ea800/0xb0ce63137acc4daf lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4db6 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787784695.358453:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787784695.358457:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9895054dd800 from 52 10000000:01000000:3.0:1787784695.358460:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787784695.358520:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b3600/0xb0ce63137acc4dbd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.358528:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff98953f5b3600/0xb0ce63137acc4dbd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.358568:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff98953f5b3600/0xb0ce63137acc4dbd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784695.358606:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366592:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.358703:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784695.358810:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1480 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.361015:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366592 00000100:00100000:2.0:1787784695.361038:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366592 00000100:00100000:2.0:1787784695.361120:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 501 00000100:00100000:2.0:1787784695.361133:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+16:5808:x1874627668366592:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784695.361485:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784695.361514:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4dc4 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4dbd expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.361552:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4dc4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4dbd expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.361565:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032ea600/0xb0ce63137acc4dc4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4dbd expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.361576:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea600, rc 0) 00000100:00100000:2.0:1787784695.361990:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668366592:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 857us (3285us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.361999:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 501 00000100:00100000:3.0:1787784695.362576:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366592:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784695.362605:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff98953f5b3600/0xb0ce63137acc4dbd lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4dc4 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.362618:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff98953f5b3600/0xb0ce63137acc4dbd lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4dc4 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784695.362870:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366720:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.362923:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784695.362970:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1480 going to sleep for 11 seconds 00000100:00100000:1.0:1787784695.362979:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366720 00000100:00100000:1.0:1787784695.363005:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366720 00000100:00100000:1.0:1787784695.363061:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:1.0:1787784695.363071:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668366720:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.363500:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668366720:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 429us (574us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.363509:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:3.0:1787784695.363542:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366720:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.363583:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366848:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.363630:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784695.363650:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1480 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.363844:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366848 00000100:00100000:2.0:1787784695.363857:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366848 00000100:00100000:2.0:1787784695.363887:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1787784695.363894:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668366848:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.364033:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668366848:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 140us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.364038:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:3.0:1787784695.364328:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668366848:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.364784:0:17923:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17923:1874627668366976:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784695.364834:0:17923:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.364908:0:17923:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784695.364948:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668366976 00000100:00100000:0.0:1787784695.364954:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668366976 00000100:00100000:0.0:1787784695.365024:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:0.0:1787784695.365036:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17923:x1874627668366976:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1787784695.365344:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+17:17923:x1874627668366976:12345-0@lo:502:llog_process_th.0 Request processed in 310us (508us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.365352:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:2.0:1787784695.365450:0:17923:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17923:1874627668366976:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.365500:0:17923:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17923:1874627668367104:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.365536:0:17923:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.369520:0:17923:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784695.369622:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668367104 00000100:00100000:1.0:1787784695.369635:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668367104 00000100:00100000:1.0:1787784695.369709:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:1.0:1787784695.369721:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:17923:x1874627668367104:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784695.370052:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:17923:x1874627668367104:12345-0@lo:502:llog_process_th.0 Request processed in 333us (4513us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.370061:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1787784695.370260:0:17923:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17923:1874627668367104:0@lo:502:llog_process_th.0 00000040:00080000:2.0:1787784695.370306:0:17923:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 55 count 54 00000020:01000000:1.0:1787784695.370551:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-54 (rc=0) 10000000:01000000:1.0:1787784695.370574:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1787784695.370587:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff98953f5b3600/0xb0ce63137acc4dbd lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4dc4 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.370599:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff98953f5b3600/0xb0ce63137acc4dbd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4dc4 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784695.370611:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:1.0:1787784695.370614:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 46 10000000:01000000:1.0:1787784695.370617:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:1.0:1787784695.370657:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4dcb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.370674:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4dcb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.370685:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4dcb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784695.370711:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367232:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.370759:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.370799:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.370841:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668367232 00000100:00100000:2.0:1787784695.370846:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668367232 00000100:00100000:2.0:1787784695.370873:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:2.0:1787784695.370879:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+17:5808:x1874627668367232:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784695.370890:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784695.370903:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032ea000/0xb0ce63137acc4dd2 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4dcb expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.370930:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032ea000/0xb0ce63137acc4dd2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4dcb expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.370941:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032ea000/0xb0ce63137acc4dd2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4dcb expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.370951:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032ea000, rc 0) 00000100:00100000:2.0:1787784695.370999:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668367232:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 119us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.371006:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1787784695.371066:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367232:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784695.371078:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4dcb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4dd2 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.371087:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4dcb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4dd2 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784695.371186:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367360:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.371213:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.371237:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.371342:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668367360 00000100:00100000:2.0:1787784695.371346:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668367360 00000100:00100000:2.0:1787784695.371364:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:2.0:1787784695.371370:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668367360:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.371662:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668367360:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 291us (447us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.371670:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:1.0:1787784695.371737:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367360:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.371767:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367488:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.371807:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.371837:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.371871:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668367488 00000100:00100000:2.0:1787784695.371875:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668367488 00000100:00100000:2.0:1787784695.371893:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:2.0:1787784695.371899:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668367488:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.372058:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668367488:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 158us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.372064:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:1.0:1787784695.372266:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367488:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.372650:0:17924:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17924:1874627668367616:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.372695:0:17924:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.372728:0:17924:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1a9240 going to sleep for 11 seconds 00000100:00100000:1.0:1787784695.372769:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668367616 00000100:00100000:1.0:1787784695.372779:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668367616 00000100:00100000:1.0:1787784695.372805:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:1.0:1787784695.372814:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:17924:x1874627668367616:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784695.373044:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:17924:x1874627668367616:12345-0@lo:502:llog_process_th.0 Request processed in 231us (348us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.373052:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00100000:2.0:1787784695.373135:0:17924:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17924:1874627668367616:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1787784695.373187:0:17924:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787784695.373201:0:17924:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787784695.373203:0:17924:0:(obd_config.c:1563:class_process_config()) marker 35 (0x1) lustre-MDT0000-m del lustre.test_11c 00000020:00000080:2.0:1787784695.373209:0:17924:0:(obd_config.c:1472:class_process_config()) processing cmd: ce023 00000100:00100000:2.0:1787784695.374747:0:17924:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17924:1874627668367744:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1787784695.374828:0:17924:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784695.374873:0:17924:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1a9180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784695.375046:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668367744 00000100:00100000:1.0:1787784695.375053:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668367744 00000100:00100000:1.0:1787784695.375093:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:1.0:1787784695.375103:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:17924:x1874627668367744:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1787784695.375346:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:17924:x1874627668367744:12345-0@lo:502:llog_process_th.0 Request processed in 243us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.375355:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00100000:2.0:1787784695.375727:0:17924:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job llog_process_th:a5a93c05-6249-4607-a205-909a326a8fd1:17924:1874627668367744:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1787784695.375765:0:17924:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787784695.375773:0:17924:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787784695.375775:0:17924:0:(obd_config.c:1563:class_process_config()) marker 35 (0x2) lustre-MDT0000-m del lustre.test_11c 00000040:00080000:2.0:1787784695.375780:0:17924:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 50 count 49 00000020:01000000:1.0:1787784695.376045:0:5808:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 46-49 (rc=0) 10000000:01000000:1.0:1787784695.376074:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:1.0:1787784695.376086:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4dcb lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4dd2 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.376099:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989403640a00/0xb0ce63137acc4dcb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4dd2 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787784695.376111:0:5808:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787784695.379363:0:5808:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff989508964d28 from 1 10000000:01000000:0.0:1787784695.379372:0:5808:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1787784695.379437:0:5808:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4dd9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.379451:0:5808:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4dd9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.379465:0:5808:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4dd9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784695.379496:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367872:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.379583:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784695.379644:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784695.379857:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668367872 00000100:00100000:2.0:1787784695.379866:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668367872 00000100:00100000:2.0:1787784695.379936:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:2.0:1787784695.379947:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+18:5808:x1874627668367872:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784695.379964:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784695.379980:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4de0 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0xb0ce63137acc4dd9 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.380011:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4de0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0xb0ce63137acc4dd9 expref: 19 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.380022:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eb200/0xb0ce63137acc4de0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0xb0ce63137acc4dd9 expref: 19 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.380033:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb200, rc 0) 00000100:00100000:2.0:1787784695.380089:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668367872:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 144us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.380095:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00000100:00100000:1.0:1787784695.380512:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668367872:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787784695.380533:0:5808:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4dd9 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4de0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.380544:0:5808:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4dd9 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0xb0ce63137acc4de0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784695.380644:0:5808:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668368000:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787784695.380681:0:5808:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784695.381838:0:5808:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32240 going to sleep for 11 seconds 00000100:00100000:3.0:1787784695.381887:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368000 00000100:00100000:3.0:1787784695.381895:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368000 00000100:00100000:3.0:1787784695.381997:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:3.0:1787784695.382008:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668368000:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787784695.382420:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:5808:x1874627668368000:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 412us (1736us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784695.382428:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:1.0:1787784695.382822:0:5808:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:a5a93c05-6249-4607-a205-909a326a8fd1:5808:1874627668368000:0@lo:501:ll_cfg_requeue.0 10000000:01000000:1.0:1787784695.382875:0:5808:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.143@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1787784695.382887:0:5808:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4dd9 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4de0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784695.382899:0:5808:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.143@tcp lock: ffff989401dbca00/0xb0ce63137acc4dd9 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0xb0ce63137acc4de0 expref: -99 pid: 5808 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784695.560635:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.560697:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.560731:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.560783:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.560803:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784695.560803:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658015488 00000100:00100000:0.0:1787784695.560815:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658015488 00000100:00100000:3.0:1787784695.560824:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.560848:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.560874:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658015744 00000100:00100000:3.0:1787784695.560877:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658015744 00000100:00100000:0.0:1787784695.560886:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658015872 00000100:00100000:0.0:1787784695.560889:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658015872 00000100:00100000:3.0:1787784695.560905:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016000 00000100:00100000:3.0:1787784695.560908:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016000 00000100:00100000:3.0:1787784695.560922:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658015616 00000100:00100000:3.0:1787784695.560924:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658015616 00000100:00100000:3.0:1787784695.560936:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016128 00000100:00100000:3.0:1787784695.560938:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016128 00000100:00100000:0.0:1787784695.560943:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:3.0:1787784695.560952:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 251 00000100:00100000:0.0:1787784695.560953:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2354:x1874627658015488:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784695.560958:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627658015744:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784695.561062:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 252 00000100:00100000:1.0:1787784695.561073:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658016000:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784695.561189:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 253 00000100:00100000:2.0:1787784695.561200:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627658015616:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784695.561302:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627658015744:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 342us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784695.561310:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 251 00000100:00100000:3.0:1787784695.561328:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 254 00000100:00100000:3.0:1787784695.561334:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658016128:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784695.565388:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:2354:x1874627658015488:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 4414us (4745us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.565409:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:0.0:1787784695.565469:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00100000:0.0:1787784695.565485:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2354:x1874627658015872:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784695.565548:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:2354:x1874627658015872:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 71us (4723us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.565555:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00080000:0.0:1787784695.565639:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784695.565647:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784695.566802:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784695.566862:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658015360 00000100:00100000:0.0:1787784695.566874:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658015360 00000100:00100000:0.0:1787784695.567305:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 513 00000100:00100000:0.0:1787784695.567318:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:2354:x1874627658015360:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784695.567390:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:2354:x1874627658015360:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 75us (6606us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.567398:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 513 00000100:00100000:1.0:1787784695.577622:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658016000:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 16533us (16871us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784695.577643:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 252 00000100:00100000:2.0:1787784695.578373:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2354:x1874627658015616:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 17169us (17562us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.578385:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 253 00000100:00100000:3.0:1787784695.578703:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658016128:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 17365us (17849us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784695.578735:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 254 00010000:00010000:1.0:1787784695.730056:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784695.881457:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784695.881691:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016256 00000100:00100000:2.0:1787784695.881704:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016256 00000100:00100000:2.0:1787784695.881778:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 514 00000100:00100000:2.0:1787784695.881789:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+10:5597:x1874627658016256:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787784695.881813:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784695.881837:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894032eb400/0xb0ce63137acc4de7 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09ca expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.881876:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894032eb400/0xb0ce63137acc4de7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09ca expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.881891:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894032eb400/0xb0ce63137acc4de7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09ca expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784695.881902:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894032eb400, rc 0) 00000100:00100000:2.0:1787784695.881984:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658016256:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 196us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.881992:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 514 00000100:00100000:0.0:1787784695.889843:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784695.892244:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016384 00000100:00100000:2.0:1787784695.892266:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016384 00000100:00100000:2.0:1787784695.892369:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 515 00000100:00100000:2.0:1787784695.892382:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658016384:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.892803:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658016384:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 422us (2973us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.892822:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 515 00000100:00100000:3.0:1787784695.894972:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784695.895610:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016512 00000100:00100000:0.0:1787784695.895623:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016512 00000100:00100000:0.0:1787784695.895699:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:0.0:1787784695.895711:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658016512:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.896011:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658016512:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 301us (1050us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.896020:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:0.0:1787784695.903040:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.903275:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016640 00000100:00100000:3.0:1787784695.903290:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016640 00000100:00100000:3.0:1787784695.903364:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:3.0:1787784695.903564:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23040:x1874627658016640:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:3.0:1787784695.904011:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23040:x1874627658016640:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 635us (982us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784695.904021:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:1.0:1787784695.912786:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784695.912900:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016768 00000100:00100000:2.0:1787784695.912913:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016768 00000100:00100000:2.0:1787784695.912987:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:2.0:1787784695.912999:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23040:x1874627658016768:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784695.913686:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:23040:x1874627658016768:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 684us (910us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.913697:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:0.0:1787784695.933642:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.933800:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658016896 00000100:00100000:3.0:1787784695.933812:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658016896 00000100:00100000:3.0:1787784695.933882:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00100000:3.0:1787784695.933893:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+11:5597:x1874627658016896:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787784695.933915:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784695.933937:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4dee lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09d1 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.933973:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4dee lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09d1 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.933985:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff98953f5b3c00/0xb0ce63137acc4dee lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09d1 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784695.933996:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3c00, rc 0) 00000100:00100000:3.0:1787784695.934078:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658016896:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 185us (453us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784695.934106:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00100000:3.0:1787784695.941247:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784695.941357:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658017024 00000100:00100000:2.0:1787784695.941370:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658017024 00000100:00100000:2.0:1787784695.941439:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 520 00000100:00100000:2.0:1787784695.941452:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658017024:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.941882:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658017024:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 431us (646us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.941891:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 520 00000100:00100000:1.0:1787784695.944704:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784695.944967:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658017152 00000100:00100000:0.0:1787784695.944981:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658017152 00000100:00100000:0.0:1787784695.945056:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 521 00000100:00100000:0.0:1787784695.945076:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658017152:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787784695.945800:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658017152:12345-192.168.203.43@tcp:503:ll_cfg_requeue.0 Request processed in 726us (1106us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.945811:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 521 00000100:00100000:0.0:1787784695.953480:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784695.957117:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658017280 00000100:00100000:2.0:1787784695.957139:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658017280 00000100:00100000:2.0:1787784695.957274:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 522 00000100:00100000:2.0:1787784695.957287:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23041:x1874627658017280:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784695.957697:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23041:x1874627658017280:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 410us (4336us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784695.957706:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 522 00000100:00100000:0.0:1787784695.968322:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784695.968626:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658017408 00000100:00100000:3.0:1787784695.968640:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658017408 00000100:00100000:3.0:1787784695.968929:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:3.0:1787784695.968943:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23041:x1874627658017408:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:3.0:1787784695.969751:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:23041:x1874627658017408:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 808us (1439us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784695.969763:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:1.0:1787784695.979024:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784695.979197:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658017536 00000100:00100000:0.0:1787784695.979223:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658017536 00000100:00100000:0.0:1787784695.979298:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 524 00000100:00100000:0.0:1787784695.979311:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+12:5597:x1874627658017536:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787784695.979335:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784695.979360:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4df5 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09d8 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.979476:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4df5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09d8 expref: 13 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.979490:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989401dbcc00/0xb0ce63137acc4df5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09d8 expref: 13 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784695.979524:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbcc00, rc 0) 00000100:00100000:0.0:1787784695.979607:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658017536:12345-192.168.203.43@tcp:101:ll_cfg_requeue.0 Request processed in 298us (1247us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784695.979615:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 524 00000100:00100000:1.0:1787784695.984350:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784695.990533:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658017664 00000100:00100000:2.0:1787784695.990555:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658017664 00000100:00100000:2.0:1787784695.990662:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 525 00000100:00100000:2.0:1787784695.990674:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658017664:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787784695.992785:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:5597:x1874627658017664:12345-192.168.203.43@tcp:501:ll_cfg_requeue.0 Request processed in 2099us (8438us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787784695.992814:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 525 00010000:00010000:0.0:1787784696.757545:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784697.332410:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668368512:0@lo:400:kworker.0 00000100:00100000:1.0:1787784697.332481:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784697.332533:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1787784697.332549:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668368128:0@lo:400:kworker.0 00000100:00100000:1.0:1787784697.332579:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784697.332604:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668368640:0@lo:400:kworker.0 00000100:00100000:1.0:1787784697.332628:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784697.332653:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668368384:0@lo:400:kworker.0 00000100:00100000:1.0:1787784697.332677:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784697.332694:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1787784697.332699:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668368256:0@lo:400:kworker.0 00000100:00100000:1.0:1787784697.332720:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784697.332733:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668368768:0@lo:400:kworker.0 00000100:00100000:1.0:1787784697.332753:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784697.332795:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368128 00000100:00100000:1.0:1787784697.332802:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368128 00000100:00100000:1.0:1787784697.332843:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:1.0:1787784697.332851:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668368128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784697.332897:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668368128:12345-0@lo:400:kworker.0 Request processed in 46us (318us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784697.332903:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000100:00100000:1.0:1787784697.333284:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668368128:0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333450:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368512 00000100:00100000:0.0:1787784697.333455:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368512 00000100:00100000:0.0:1787784697.333491:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368768 00000100:00100000:0.0:1787784697.333494:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368768 00000100:00100000:0.0:1787784697.333517:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1787784697.333525:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668368512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333572:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668368512:12345-0@lo:400:kworker.0 Request processed in 49us (1091us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784697.333579:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1787784697.333597:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:0.0:1787784697.333603:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668368768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333635:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668368768:12345-0@lo:400:kworker.0 Request processed in 33us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784697.333641:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:0.0:1787784697.333658:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368640 00000100:00100000:0.0:1787784697.333661:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368640 00000100:00100000:0.0:1787784697.333685:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368384 00000100:00100000:0.0:1787784697.333688:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368384 00000100:00100000:0.0:1787784697.333700:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368256 00000100:00100000:0.0:1787784697.333702:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368256 00000100:00100000:0.0:1787784697.333713:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1787784697.333719:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668368640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333758:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668368640:12345-0@lo:400:kworker.0 Request processed in 39us (1129us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784697.333764:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1787784697.333794:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:0.0:1787784697.333801:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668368384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333830:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668368384:12345-0@lo:400:kworker.0 Request processed in 30us (1152us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784697.333836:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:0.0:1787784697.333847:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:0.0:1787784697.333852:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668368256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333878:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668368256:12345-0@lo:400:kworker.0 Request processed in 25us (1157us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784697.333883:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00080000:0.0:1787784697.333916:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784697.333920:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784697.333922:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784697.333955:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668368640:0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333967:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668368512:0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.333993:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668368256:0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.334003:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668368768:0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.334012:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668368384:0@lo:400:kworker.0 00000100:00100000:0.0:1787784697.340350:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668368896:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784697.340417:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784697.340531:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668368896 00000100:00100000:2.0:1787784697.340540:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668368896 00000100:00100000:2.0:1787784697.340593:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1787784697.340603:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668368896:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784697.340647:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784697.340696:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668368896:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784697.340702:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1787784697.340740:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668368896:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1787784697.778480:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787784698.019831:0:17973:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 18:51:36 (1787784696) 00000100:00100000:3.0:1787784698.353001:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784698.353231:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658017920 00000100:00100000:0.0:1787784698.353247:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658017920 00000100:00100000:0.0:1787784698.353358:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 744 00000100:00100000:0.0:1787784698.353369:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+31:23232:x1874627658017920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784698.353399:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784698.353432:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4dfc lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09df expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.353831:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4e03 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.353850:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4e03 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.353868:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4e03 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784698.353881:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:0.0:1787784698.354192:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4dfc lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09df expref: 34 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.354225:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbdc00/0xb0ce63137acc4e03 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09df expref: 33 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.354236:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784698.354240:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787784698.354244:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbdc00, rc 0) 00000100:00100000:0.0:1787784698.354330:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+33:23232:x1874627658017920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 962us (1360us total) trans 4294967786 rc 0/0 00000100:00100000:0.0:1787784698.354353:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 744 00000100:00080000:2.0:1787784698.354605:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784698.361759:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784698.361824:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658018048 00000100:00100000:1.0:1787784698.361838:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658018048 00000100:00100000:1.0:1787784698.361922:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 745 00000100:00100000:1.0:1787784698.361936:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+33:23232:x1874627658018048:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1787784698.362093:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+33:23232:x1874627658018048:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 159us (346us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784698.362118:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 745 00000100:00100000:1.0:1787784698.378641:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784698.384955:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658018176 00000100:00100000:3.0:1787784698.384979:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658018176 00000100:00100000:3.0:1787784698.385102:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 746 00000100:00100000:3.0:1787784698.385116:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:23233:x1874627658018176:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784698.385183:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784698.385223:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4e11 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09e6 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.385319:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.385338:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.385352:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.385398:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.385406:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.385413:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787784698.385861:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784698.386941:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.386960:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.386969:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.386976:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784698.386981:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.386991:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387011:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:3.0:1787784698.387014:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784698.387020:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387027:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4e18 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387066:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387073:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387079:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387085:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784698.387088:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387096:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387104:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff989505542400) 00010000:00010000:3.0:1787784698.387107:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784698.387110:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387116:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4e1f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387225:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4e11 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09e6 expref: 75 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387237:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:3.0:1787784698.387239:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784698.387250:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4e11 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09e6 expref: 75 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.387262:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784698.387356:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:23233:x1874627658018176:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2242us (8727us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784698.387366:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 746 00000100:00100000:0.0:1787784698.400833:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784698.401016:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658018432 00000100:00100000:1.0:1787784698.401030:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658018432 00000100:00100000:1.0:1787784698.401110:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 747 00000100:00100000:1.0:1787784698.401123:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:23233:x1874627658018432:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:1.0:1787784698.401256:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784698.401291:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4c82 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b098b expref: 74 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.401317:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:1.0:1787784698.401323:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784698.401673:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.401692:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 32 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.401702:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.401886:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 32 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.401903:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 32 pid: 5811 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784698.401921:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668369024:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784698.402056:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784698.404246:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668369024:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784698.404286:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784698.405031:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784698.405123:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658018560 00000100:00100000:1.0:1787784698.405132:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658018560 00010000:00010000:1.0:1787784698.405225:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 33 pid: 5811 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784698.405262:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 244 00000100:00100000:1.0:1787784698.405274:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+33:2356:x1874627658018560:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784698.405293:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784698.405301:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 33 pid: 5811 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.405314:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 33 pid: 5811 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.405342:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4cac lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0999 expref: 33 pid: 5811 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.405362:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:1.0:1787784698.405367:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.405377:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.405407:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.405446:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784698.405517:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+32:2356:x1874627658018560:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 244us (488us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784698.405526:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 244 00010000:00010000:2.0:1787784698.405734:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.405744:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.405776:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.405786:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.405795:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784698.407188:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 512 00010000:00010000:2.0:1787784698.407219:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.407230:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.407236:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.407243:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784698.407248:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.407257:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.407273:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895029d1900) 00010000:00010000:2.0:1787784698.407276:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784698.407283:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.407290:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4e26 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784698.407311:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e07bd21b reply state 00000000944748b9 transno 4294967787 ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784698.407321:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784698.407440:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:23233:x1874627658018432:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 6314us (6618us total) trans 4294967787 rc 0/0 00000100:00100000:2.0:1787784698.407463:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 747 00010000:00010000:3.0:1787784698.414011:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.414035:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.414044:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.414051:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784698.414057:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.414068:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.414089:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff989530cb1900) 00010000:00010000:3.0:1787784698.414096:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784698.414102:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.414110:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e2d lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784698.768060:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784698.768789:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658018688 00000100:00100000:0.0:1787784698.768805:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658018688 00000100:00100000:0.0:1787784698.768957:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 748 00000100:00100000:0.0:1787784698.768972:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:23270:x1874627658018688:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784698.769008:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784698.769042:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e34 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09ed expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.769128:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e3b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.769172:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.769187:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784698.769961:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784698.769967:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 21 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784698.770008:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.770045:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e34 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09ed expref: 76 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.770098:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e3b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09ed expref: 75 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.770107:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d800) 00010000:00010000:0.0:1787784698.770110:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784698.770114:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd000, rc 0) 00000100:00100000:0.0:1787784698.772455:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:23270:x1874627658018688:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 3468us (4396us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784698.772473:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 748 00000100:00100000:0.0:1787784698.780726:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784698.780835:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658018816 00000100:00100000:2.0:1787784698.780848:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658018816 00000100:00100000:2.0:1787784698.780948:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 749 00000100:00100000:2.0:1787784698.780961:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:2356:x1874627658018816:12345-192.168.203.43@tcp:41:mkdir.0 00000004:00000024:2.0:1787784698.781025:0:5812:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 2259812352 avail 2257715200 00000100:00100000:2.0:1787784698.781117:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:2356:x1874627658018816:12345-192.168.203.43@tcp:41:mkdir.0 Request processed in 159us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784698.781125:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 749 00000100:00100000:3.0:1787784698.784490:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784698.784799:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658019072 00000100:00100000:0.0:1787784698.784813:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658019072 00000100:00100000:0.0:1787784698.784896:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 750 00000100:00100000:0.0:1787784698.784915:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:23270:x1874627658019072:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:0.0:1787784698.784993:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784698.785031:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e3b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09ed expref: 75 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.785056:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d800) 00010000:00010000:0.0:1787784698.785069:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784698.785602:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:23270:x1874627658019072:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 687us (1124us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787784698.785611:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 750 00000100:00100000:0.0:1787784698.789406:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784698.789722:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658019200 00000100:00100000:2.0:1787784698.789736:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658019200 00000100:00100000:2.0:1787784698.790609:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 751 00000100:00100000:2.0:1787784698.790630:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:23270:x1874627658019200:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:2.0:1787784698.790660:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784698.790694:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4e42 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09f4 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.791328:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e49 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.791352:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.791366:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784698.791453:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784698.791477:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 21 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784698.791509:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.791540:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4e42 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09f4 expref: 76 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.791561:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e49 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09f4 expref: 75 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.791571:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784698.791574:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784698.791578:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905200, rc 0) 00000100:00100000:2.0:1787784698.791716:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:23270:x1874627658019200:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 1093us (2322us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784698.791726:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 751 00000100:00100000:1.0:1787784698.800483:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784698.800603:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658019328 00000100:00100000:0.0:1787784698.800636:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658019328 00000100:00100000:0.0:1787784698.800751:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 752 00000100:00100000:0.0:1787784698.800764:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:23270:x1874627658019328:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784698.800794:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784698.800832:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e50 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b09fb expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.800932:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.800951:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.800965:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.800982:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.800990:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.800997:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801140:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4e65 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801174:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4e65 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801182:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4e65 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784698.801272:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 512 00000004:00200000:0.0:1787784698.801277:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784698.801306:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4e65 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801322:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801329:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801336:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801342:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784698.801347:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801356:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801369:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:0.0:1787784698.801372:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784698.801377:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801383:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4e57 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801398:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801442:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801448:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801453:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784698.801456:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801462:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801472:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff989503a6d800) 00010000:00010000:0.0:1787784698.801474:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784698.801477:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801482:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4e5e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801519:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4e50 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09fb expref: 77 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801534:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4e65 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09fb expref: 76 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.801541:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784698.801544:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784698.801552:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc400, rc 0) 00000100:00100000:0.0:1787784698.801643:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23270:x1874627658019328:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 882us (1174us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784698.801651:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 752 00010000:00010000:1.0:1787784698.802763:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784698.868580:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784698.868761:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658019456 00000100:00100000:1.0:1787784698.868775:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658019456 00000100:00100000:1.0:1787784698.868927:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 753 00000100:00100000:1.0:1787784698.868940:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+32:23276:x1874627658019456:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784698.868971:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784698.869010:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4e6c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a02 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.869113:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.869130:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.869144:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784698.869284:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784698.869289:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 21 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784698.869333:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.869386:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4e6c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 34 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.869406:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.869415:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784698.869419:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784698.869422:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641400, rc 0) 00000100:00100000:1.0:1787784698.869515:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+33:23276:x1874627658019456:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 576us (951us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784698.869522:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 753 00000100:00100000:3.0:1787784698.877866:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784698.878099:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658019712 00000100:00100000:2.0:1787784698.878112:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658019712 00000100:00100000:2.0:1787784698.878229:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 754 00000100:00100000:2.0:1787784698.878241:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23276:x1874627658019712:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784698.878269:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784698.878274:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787784698.878428:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4e65 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09fb expref: 76 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.878456:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784698.878469:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784698.878491:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4e7a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a09 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.878764:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.878780:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.878791:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.878805:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.878812:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.878819:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880503:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880523:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880536:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784698.880549:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 0 00010000:00010000:2.0:1787784698.880577:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880600:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880607:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880613:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784698.880618:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880626:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.880640:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784698.880642:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784698.881074:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.881084:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc4e81 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784698.881102:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cf354311 reply state 000000008023be2e transno 4294967788 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784698.881112:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784698.881519:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc4e7a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 78 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.881545:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 77 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.881572:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989530cb1300) 00010000:00010000:2.0:1787784698.881575:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784698.881578:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc800, rc 0) 00000100:00100000:2.0:1787784698.881755:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23276:x1874627658019712:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3513us (3903us total) trans 4294967788 rc 0/0 00000100:00100000:2.0:1787784698.881766:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 754 00000100:00100000:2.0:1787784698.887497:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784698.887635:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658019968 00000100:00100000:1.0:1787784698.887648:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658019968 00000100:00100000:1.0:1787784698.887737:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 755 00000100:00100000:1.0:1787784698.887750:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23276:x1874627658019968:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784698.887795:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784698.887827:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4e9d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a10 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.887974:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.887991:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 79 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888001:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888064:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 79 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888079:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 79 pid: 5813 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784698.888096:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668369152:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784698.888464:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1c00 going to sleep for 11 seconds 00010000:00010000:1.0:1787784698.888517:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888527:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888534:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888540:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784698.888545:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888554:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888571:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff989530cb1700) 00010000:00010000:1.0:1787784698.888574:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784698.888579:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.888598:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc4e88 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784698.891990:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668369152:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784698.892030:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784698.894978:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784698.895632:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658020096 00000100:00100000:1.0:1787784698.895646:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658020096 00010000:00010000:1.0:1787784698.895700:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 79 pid: 5813 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784698.895753:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:1.0:1787784698.895764:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:2356:x1874627658020096:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784698.895957:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784698.895968:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 79 pid: 5813 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.895984:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 79 pid: 5813 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.896012:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc4e8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a09 expref: 79 pid: 5813 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.896032:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989530cb1300) 00010000:00010000:1.0:1787784698.896040:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.896069:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.896079:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.896233:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784698.896432:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:2356:x1874627658020096:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 669us (1470us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784698.896442:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 245 00010000:00010000:0.0:1787784698.898979:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.899000:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784698.899030:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 0 00010000:00010000:0.0:1787784698.899647:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.899660:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.899667:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.899746:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784698.899751:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.899760:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.899849:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989530cb1300) 00010000:00010000:0.0:1787784698.899853:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784698.899859:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.899960:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ea4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.900053:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4e9d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a10 expref: 78 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.900067:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:0.0:1787784698.900069:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784698.900073:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc4e9d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a10 expref: 78 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.900087:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641e00, rc 0) 00000100:00100000:0.0:1787784698.900599:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23276:x1874627658019968:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 12849us (13116us total) trans 4294967789 rc 301/301 00000100:00100000:0.0:1787784698.900612:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 755 00000100:00100000:2.0:1787784698.904845:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784698.905050:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658020352 00000100:00100000:3.0:1787784698.905062:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658020352 00000100:00100000:3.0:1787784698.905220:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 154 00000100:00100000:3.0:1787784698.905233:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23276:x1874627658020352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784698.905745:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23276:x1874627658020352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 513us (914us total) trans 4294967790 rc 0/0 00000100:00100000:3.0:1787784698.905756:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 154 00000100:00100000:2.0:1787784698.909996:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784698.910098:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658020480 00000100:00100000:0.0:1787784698.910112:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658020480 00000100:00100000:0.0:1787784698.910448:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 756 00000100:00100000:0.0:1787784698.910461:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23276:x1874627658020480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784698.910495:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784698.910533:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a17 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.910612:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 79 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.910626:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 79 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.910662:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:0.0:1787784698.910664:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784698.910667:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784698.910746:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:23276:x1874627658020480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 287us (763us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784698.910754:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 756 00000100:00100000:0.0:1787784698.913647:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784698.914099:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658020736 00000100:00100000:2.0:1787784698.914111:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658020736 00000100:00100000:2.0:1787784698.914238:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 757 00000100:00100000:2.0:1787784698.914251:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:23276:x1874627658020736:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1787784698.914521:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.914543:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.914560:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.914985:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.914995:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.915021:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.915028:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fde00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784698.915033:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.915042:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.915057:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:2.0:1787784698.915061:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787784698.915066:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.915073:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4eb9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784698.915543:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23276:x1874627658020736:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 1289us (1907us total) trans 4294967791 rc 0/0 00000100:00100000:2.0:1787784698.915556:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 757 00000100:00100000:2.0:1787784698.918483:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784698.918586:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658020992 00000100:00100000:3.0:1787784698.918598:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658020992 00000100:00100000:3.0:1787784698.918674:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 758 00000100:00100000:3.0:1787784698.918686:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23276:x1874627658020992:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784698.919653:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.919682:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784698.919697:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784698.920525:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967792 00000004:00010000:3.0:1787784698.920631:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002bcc85b0 reply state 00000000c25f6e34 transno 4294967793 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784698.920667:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784698.921077:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:23276:x1874627658020992:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 2385us (2600us total) trans 4294967793 rc 0/0 00000100:00100000:3.0:1787784698.921088:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 758 00000020:00080000:3.0:1787784698.921096:0:5813:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1787784698.924815:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.924838:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.924847:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.924873:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784698.924879:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.924897:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.924920:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:2.0:1787784698.924924:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787784698.924930:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784698.924936:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4ec0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784698.925369:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784698.925579:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658021248 00000100:00100000:2.0:1787784698.925594:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658021248 00000100:00100000:2.0:1787784698.925658:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 155 00000100:00100000:2.0:1787784698.925669:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950767a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:23276:x1874627658021248:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784698.926560:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950767a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:23276:x1874627658021248:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 886us (1407us total) trans 4294967794 rc 0/0 00000100:00100000:2.0:1787784698.926573:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 155 00000100:00100000:0.0:1787784698.975576:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784698.975673:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658021376 00000100:00100000:1.0:1787784698.975688:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658021376 00000100:00100000:1.0:1787784698.975794:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 759 00000100:00100000:1.0:1787784698.975806:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+33:23279:x1874627658021376:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784698.975848:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784698.975883:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ec7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a1e expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.976950:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.976976:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.976991:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.977012:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.977044:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.977052:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.980895:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4edc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.980926:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4edc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.980942:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4edc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784698.981000:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 512 00000004:00200000:1.0:1787784698.981005:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784698.981034:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4edc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981050:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981057:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981065:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981072:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784698.981077:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981104:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981119:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784698.981122:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784698.981127:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981132:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc4ece lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981171:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981178:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981184:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981190:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784698.981203:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981210:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981218:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff9895029d1c00) 00010000:00010000:1.0:1787784698.981220:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784698.981223:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981240:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc4ed5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.981294:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc4ec7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a1e expref: 35 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.982504:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640e00/0xb0ce63137acc4edc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a1e expref: 34 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784698.982516:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:1.0:1787784698.982520:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784698.982523:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640e00, rc 0) 00000100:00100000:1.0:1787784698.982866:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23279:x1874627658021376:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7058us (7304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784698.982877:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 759 00000100:00100000:1.0:1787784698.989419:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784698.995310:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658021632 00000100:00100000:0.0:1787784698.995333:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658021632 00000100:00100000:0.0:1787784698.995442:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 760 00000100:00100000:0.0:1787784698.995456:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23279:x1874627658021632:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784698.995963:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784698.996000:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4ee3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a25 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.996113:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 3/1,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.996131:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.997396:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.997447:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.997764:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.998059:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.998666:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4ef8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.998744:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4ef8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.998895:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4ef8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784698.998920:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:0.0:1787784698.999060:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999069:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999077:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999083:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784698.999089:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999097:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999132:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:0.0:1787784698.999135:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784698.999141:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999855:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc4eea lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999874:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999879:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999885:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999890:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784698.999893:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999900:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999910:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff989503a6da00) 00010000:00010000:0.0:1787784698.999913:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784698.999916:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784698.999921:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc4ef1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.000007:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4ee3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a25 expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.000023:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4ef8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a25 expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.000030:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:0.0:1787784699.000032:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784699.000035:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16000, rc 0) 00000100:00100000:0.0:1787784699.000142:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23279:x1874627658021632:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4687us (10737us total) trans 4294967795 rc 0/0 00000100:00100000:0.0:1787784699.000506:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 760 00000100:00100000:3.0:1787784699.010622:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.011948:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658021760 00000100:00100000:2.0:1787784699.011968:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658021760 00000100:00100000:2.0:1787784699.012067:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 761 00000100:00100000:2.0:1787784699.012085:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23279:x1874627658021760:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:2.0:1787784699.012258:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012299:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012315:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012354:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012363:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012370:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012377:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fde00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784699.012382:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012391:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012405:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:2.0:1787784699.012408:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784699.012414:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.012421:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f06 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.012551:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23279:x1874627658021760:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 469us (1944us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.012566:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 761 00000100:00100000:3.0:1787784699.017246:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.017389:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658022016 00000100:00100000:2.0:1787784699.017403:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658022016 00000100:00100000:2.0:1787784699.017479:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 156 00000100:00100000:2.0:1787784699.017492:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23279:x1874627658022016:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:2.0:1787784699.017968:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23279:x1874627658022016:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 475us (735us total) trans 4294967796 rc 0/0 00000100:00100000:2.0:1787784699.017978:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 156 00000100:00100000:3.0:1787784699.095719:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.096219:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658022144 00000100:00100000:2.0:1787784699.096234:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658022144 00000100:00100000:2.0:1787784699.096321:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 762 00000100:00100000:2.0:1787784699.096335:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23287:x1874627658022144:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784699.096372:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784699.096402:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f0d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a2c expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.096531:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc4f14 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.096550:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc4f14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.096565:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc4f14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784699.096649:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 512 00000004:00200000:2.0:1787784699.096654:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784699.096695:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc4f14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.096728:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f0d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a2c expref: 39 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.096750:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc4f14 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a2c expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.096761:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:2.0:1787784699.096764:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784699.096790:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784699.096887:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23287:x1874627658022144:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 554us (1183us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.096895:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 762 00000100:00100000:0.0:1787784699.102220:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784699.102325:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658022400 00000100:00100000:1.0:1787784699.102338:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658022400 00000100:00100000:1.0:1787784699.102423:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 31 00000100:00100000:1.0:1787784699.102435:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23287:x1874627658022400:12345-192.168.203.43@tcp:103:lfs.0 00010000:00010000:1.0:1787784699.102457:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784699.102493:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc4ef8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a25 expref: 38 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.102517:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:1.0:1787784699.102522:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784699.102616:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23287:x1874627658022400:12345-192.168.203.43@tcp:103:lfs.0 Request processed in 183us (422us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784699.102625:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 31 00000100:00100000:3.0:1787784699.109098:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.109259:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658022528 00000100:00100000:2.0:1787784699.109274:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658022528 00000100:00100000:2.0:1787784699.109484:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 763 00000100:00100000:2.0:1787784699.109498:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23287:x1874627658022528:12345-192.168.203.43@tcp:58:lfs.0 00010000:00010000:2.0:1787784699.109648:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.109672:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.109684:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 4/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.109731:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.109769:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 82 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.109789:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668369280:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784699.111455:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cf3e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784699.115422:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.115570:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668369280:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784699.115607:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784699.115699:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658022656 00000100:00100000:1.0:1787784699.115710:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658022656 00010000:00010000:1.0:1787784699.115758:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 83 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784699.115808:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 246 00000100:00100000:1.0:1787784699.115820:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:2356:x1874627658022656:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784699.115840:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784699.115847:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 83 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.115860:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 83 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.115888:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4eb2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a17 expref: 83 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.115908:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:1.0:1787784699.115915:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.115925:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.115933:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.115992:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784699.116057:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:2356:x1874627658022656:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 239us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784699.116066:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 246 00010000:00010000:3.0:1787784699.116770:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.116943:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784699.117291:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].18, 136 off10640 00000100:00100000:3.0:1787784699.117399:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23287:x1874627658022528:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 7899us (8312us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784699.117412:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 763 00000040:00080000:0.0:1787784699.313536:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784699.313548:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784699.313628:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 19 count 2 00000040:00080000:0.0:1787784699.313636:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784699.313941:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668369408:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784699.314084:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784699.320100:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784699.320408:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668369408:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784699.320628:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658022784 00000100:00100000:3.0:1787784699.320637:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658022784 00000100:00100000:3.0:1787784699.320700:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 764 00000100:00100000:3.0:1787784699.320712:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23297:x1874627658022784:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784699.320754:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784699.321626:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23297:x1874627658022784:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 911us (1534us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784699.321637:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 764 00000100:00100000:3.0:1787784699.326984:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784699.327088:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658022912 00000100:00100000:0.0:1787784699.327101:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658022912 00000100:00100000:0.0:1787784699.327214:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 765 00000100:00100000:0.0:1787784699.327227:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23297:x1874627658022912:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1787784699.327338:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.327381:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.327398:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330738:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330763:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330777:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330813:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330822:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330830:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784699.330876:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00000004:00200000:0.0:1787784699.330882:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784699.330896:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330909:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330931:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330938:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330945:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784699.330950:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330959:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330973:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784699.330978:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784699.330984:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.330991:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc4f22 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331003:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331010:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331016:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331021:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784699.331032:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331039:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331048:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x43be2ce7 (ffff989503a6da00) 00010000:00010000:0.0:1787784699.331050:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784699.331053:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331059:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc4f29 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331075:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331081:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331086:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331092:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784699.331095:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331101:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331107:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:0.0:1787784699.331110:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784699.331119:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.331126:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc4f30 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784699.331291:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23297:x1874627658022912:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 4052us (4308us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784699.331302:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 765 00000100:00100000:1.0:1787784699.343666:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784699.343751:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658023168 00000100:00100000:0.0:1787784699.343767:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658023168 00000100:00100000:0.0:1787784699.343844:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 766 00000100:00100000:0.0:1787784699.343858:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23297:x1874627658023168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784699.343880:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784699.343902:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4f37 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a33 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344024:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344040:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344052:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784699.344062:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:0.0:1787784699.344208:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344216:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344237:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344244:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784699.344248:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344256:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.344270:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:0.0:1787784699.344272:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784699.344276:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.345040:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc4f3e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.345098:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4f37 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a33 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.345112:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784699.345116:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784699.345121:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc4f37 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a33 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.345134:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:1.0:1787784699.345883:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23297:x1874627658023168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2018us (2221us total) trans 4294967797 rc 301/301 00000100:00100000:1.0:1787784699.345898:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 766 00000100:00100000:2.0:1787784699.365100:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.365176:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658023296 00000100:00100000:2.0:1787784699.365202:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658023296 00000100:00100000:2.0:1787784699.365275:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 767 00000100:00100000:2.0:1787784699.365286:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23297:x1874627658023296:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784699.365336:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784699.365478:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23297:x1874627658023296:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 193us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.365487:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 767 00000100:00100000:2.0:1787784699.382088:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.382958:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658023552 00000100:00100000:3.0:1787784699.382975:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658023552 00000100:00100000:3.0:1787784699.383061:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 768 00000100:00100000:3.0:1787784699.383073:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23297:x1874627658023552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784699.383101:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784699.383105:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787784699.383141:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc4f14 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a2c expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.383214:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:3.0:1787784699.383221:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784699.383238:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4f4c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a3a expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784699.385546:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xf:0x0], index 0: 0 00010000:00010000:2.0:1787784699.386433:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc4f53 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.386457:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc4f53 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.386487:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc4f53 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.386579:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4f4c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a3a expref: 40 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.386603:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc4f53 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a3a expref: 39 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.386613:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784699.386616:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784699.386621:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784699.386976:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23297:x1874627658023552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3902us (4901us total) trans 4294967798 rc 0/0 00000100:00100000:2.0:1787784699.386987:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 768 00000100:00100000:2.0:1787784699.407898:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.407980:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658023808 00000100:00100000:3.0:1787784699.407994:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658023808 00000100:00100000:3.0:1787784699.408094:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 769 00000100:00100000:3.0:1787784699.408106:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23297:x1874627658023808:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784699.408226:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784699.408261:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc4f53 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a3a expref: 39 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408284:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1400) 00010000:00010000:3.0:1787784699.408314:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784699.408342:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408358:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408369:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408384:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408390:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408396:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408500:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408508:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408515:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408521:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.408525:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408533:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408548:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:3.0:1787784699.408550:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784699.408554:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408561:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f61 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408572:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408577:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408583:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408589:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.408591:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408597:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408603:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x97c2ecad (ffff989505542400) 00010000:00010000:3.0:1787784699.408605:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784699.408607:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.408613:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f68 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x97c2ecad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.408985:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23297:x1874627658023808:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 878us (1095us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784699.408993:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 769 00000100:00100000:0.0:1787784699.419988:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.422912:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658023936 00000100:00100000:3.0:1787784699.422934:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658023936 00000100:00100000:2.0:1787784699.423111:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 770 00000100:00100000:2.0:1787784699.423124:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23297:x1874627658023936:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784699.423182:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784699.423228:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4f6f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a41 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.423272:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.423288:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.423314:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.423439:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4f6f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 40 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.423457:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.423467:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784699.423470:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784699.423474:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784699.423562:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23297:x1874627658023936:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 440us (3588us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.423571:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 770 00000100:00100000:3.0:1787784699.426819:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784699.430288:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658024192 00000100:00100000:0.0:1787784699.430323:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658024192 00000100:00100000:0.0:1787784699.430405:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 771 00000100:00100000:0.0:1787784699.430432:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23297:x1874627658024192:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784699.430560:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.430589:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.430604:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784699.431048:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].6, 88 off8600 00000004:00080000:0.0:1787784699.431060:0:5813:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 490517911589ns, next at 491517908838ns, rc = 0 00000004:00010000:0.0:1787784699.431274:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008a960cf6 reply state 0000000040f0c096 transno 4294967799 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784699.431285:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784699.431426:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23297:x1874627658024192:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 993us (4616us total) trans 4294967799 rc 0/0 00000100:00100000:0.0:1787784699.431434:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 771 00010000:00010000:0.0:1787784699.433376:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.433396:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.433405:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.433411:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784699.433427:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.433437:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.433459:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:0.0:1787784699.433462:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784699.433469:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.433476:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc4f7d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784699.435801:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.436017:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658024320 00000100:00100000:3.0:1787784699.436031:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658024320 00000100:00100000:3.0:1787784699.436108:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 772 00000100:00100000:3.0:1787784699.436120:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:23297:x1874627658024320:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784699.436200:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784699.436228:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a48 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.436314:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 41 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.436330:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 41 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.436351:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:3.0:1787784699.436354:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784699.436358:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2e00, rc 0) 00000100:00100000:3.0:1787784699.436429:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23297:x1874627658024320:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 311us (642us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784699.436437:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 772 00000100:00100000:1.0:1787784699.439556:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.439645:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658024448 00000100:00100000:3.0:1787784699.439659:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658024448 00000100:00080000:0.0:1787784699.439963:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784699.440009:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:3.0:1787784699.440024:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658024448:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1787784699.440559:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658024448:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 537us (1016us total) trans 4294967328 rc 0/0 00000100:00100000:3.0:1787784699.440568:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:1.0:1787784699.443629:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.443723:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658024704 00000100:00100000:3.0:1787784699.443735:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658024704 00000100:00100000:3.0:1787784699.443917:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 157 00000100:00100000:3.0:1787784699.443928:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23297:x1874627658024704:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784699.444282:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23297:x1874627658024704:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 355us (667us total) trans 4294967800 rc 0/0 00000100:00100000:3.0:1787784699.444290:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 157 00000100:00100000:1.0:1787784699.446947:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.447409:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658024960 00000100:00100000:3.0:1787784699.447422:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658024960 00000100:00100000:3.0:1787784699.447480:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 773 00000100:00100000:3.0:1787784699.447489:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23297:x1874627658024960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784699.447631:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23297:x1874627658024960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 142us (694us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784699.447640:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 773 00000100:00100000:1.0:1787784699.451980:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.452605:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658025088 00000100:00100000:3.0:1787784699.452620:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658025088 00000100:00100000:3.0:1787784699.452711:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 774 00000100:00100000:3.0:1787784699.452723:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23297:x1874627658025088:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784699.452762:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784699.452789:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f8b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a4f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.452887:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4f92 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.452905:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4f92 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.452919:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4f92 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784699.452979:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784699.453004:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4f92 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.453037:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4f8b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a4f expref: 44 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.453057:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4f92 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a4f expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.453067:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:3.0:1787784699.453070:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1787784699.453073:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2a00, rc 0) 00000100:00100000:3.0:1787784699.453341:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23297:x1874627658025088:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 467us (1225us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784699.453351:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 774 00000100:00100000:1.0:1787784699.458423:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784699.461016:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658025216 00000100:00100000:0.0:1787784699.461035:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658025216 00000100:00100000:2.0:1787784699.461425:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:2.0:1787784699.461439:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658025216:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784699.461463:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784699.461490:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a56 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784699.461526:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a8f6dfa, blocks: 0x1 00010000:00010000:2.0:1787784699.461551:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=490s v=(0 0) 00010000:00010000:2.0:1787784699.461556:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784699.461567:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.461610:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.461625:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fda00, rc 0) 00000100:00100000:2.0:1787784699.463689:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658025216:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2243us (5269us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.463705:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:3.0:1787784699.465620:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.465702:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658025472 00000100:00100000:2.0:1787784699.465715:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658025472 00000100:00100000:2.0:1787784699.465779:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 775 00000100:00100000:2.0:1787784699.465789:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23297:x1874627658025472:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784699.465915:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23297:x1874627658025472:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 128us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.465922:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 775 00000100:00100000:1.0:1787784699.472467:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784699.472624:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658025600 00000100:00100000:0.0:1787784699.472637:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658025600 00000100:00100000:0.0:1787784699.472710:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 191 00000100:00100000:0.0:1787784699.472723:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:23297:x1874627658025600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784699.472748:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784699.472774:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a5d expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.472829:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.472842:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784699.472847:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784699.472871:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 4/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.472920:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784699.472937:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 11 pid: 6671 timeout: 590 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784699.472956:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668369536:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784699.473075:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051df00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784699.478571:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.479277:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658025728 00000100:00100000:3.0:1787784699.479292:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658025728 00010000:00010000:3.0:1787784699.479346:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 12 pid: 6671 timeout: 590 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.479400:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 247 00000100:00100000:3.0:1787784699.479411:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ef050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658025728:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784699.479432:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784699.479463:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 12 pid: 6671 timeout: 590 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.479479:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 12 pid: 6671 timeout: 590 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.479514:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784699.479519:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784699.479530:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.479541:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.479590:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.479607:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668369664:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784699.479723:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d0c0 going to sleep for 11 seconds 00010000:00010000:2.0:1787784699.483651:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4f99 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a56 expref: 12 pid: 6671 timeout: 590 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.483679:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668369536:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784699.483720:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.483739:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd600, rc 0) 00000100:00100000:2.0:1787784699.483868:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:23297:x1874627658025600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 11145us (11412us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.483889:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 191 00000100:00100000:3.0:1787784699.491051:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668369664:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784699.491088:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784699.491183:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ef050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658025728:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 11771us (12625us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784699.491442:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.491959:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 247 00000100:00100000:3.0:1787784699.492090:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658025856 00000100:00100000:3.0:1787784699.492101:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658025856 00002000:00100000:3.0:1787784699.492471:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (0->157) req@ffff989518f0ad00 x1874627658025856/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:580/0 lens 648/0 e 0 to 0 dl 1787784710 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784699.492484:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f0ad00 with x1874627658025856 ext(0->157) 00002000:00010000:3.0:1787784699.492490:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989518f0ad00 00000100:00100000:3.0:1787784699.492519:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 61 00000100:00100000:3.0:1787784699.492530:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658025856:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1787784699.493557:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658025856:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 1022us (2111us total) trans 4294967329 rc 0/0 00000100:00100000:3.0:1787784699.493567:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 61 00002000:00100000:3.0:1787784699.493577:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (0->157) req@ffff989518f0ad00 x1874627658025856/t4294967329(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:580/0 lens 648/448 e 0 to 0 dl 1787784710 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784699.493585:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f0ad00 with x1874627658025856 ext(0->157) 00002000:00010000:3.0:1787784699.493589:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989518f0ad00 00000100:00100000:0.0:1787784699.498337:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.498456:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658025984 00000100:00100000:2.0:1787784699.498470:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658025984 00000100:00100000:2.0:1787784699.498557:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 776 00000100:00100000:2.0:1787784699.498569:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23297:x1874627658025984:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1787784699.592220:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23297:x1874627658025984:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 93635us (93878us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.592240:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 776 00000001:00080000:2.0:1787784699.592308:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967799 is committed 00000004:00080000:2.0:1787784699.592318:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 7 committed 00000001:00080000:2.0:1787784699.592345:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967789 is committed 00000001:00080000:2.0:1787784699.592360:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967794 is committed 00000001:00080000:2.0:1787784699.592376:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967800 is committed 00000100:00100000:2.0:1787784699.592576:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668369792:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1787784699.592670:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784699.592764:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668369792 00000100:00100000:2.0:1787784699.592785:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668369792 00000100:00100000:2.0:1787784699.592858:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1787784699.592870:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668369792:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1787784699.593077:0:10688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:2.0:1787784699.593082:0:10688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb mtime from disk: 0 -> 1787784698 00002000:00010000:2.0:1787784699.593085:0:10688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1787784699.593087:0:10688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb ctime from disk: 1787784698 -> 1787784698 00002000:00010000:2.0:1787784699.593090:0:10688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1787784699.593130:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668369792:12345-0@lo:2:osp-syn-0-0.0 Request processed in 263us (461us total) trans 4294967330 rc 0/0 00000100:00100000:2.0:1787784699.593170:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00080000:2.0:1787784699.593203:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:2.0:1787784699.593250:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9894011ecb40/2, rc 0, transno 4294967330 00000100:00100000:2.0:1787784699.593256:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668369792:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1787784699.595767:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.595860:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658026112 00000100:00100000:2.0:1787784699.595873:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658026112 00000100:00100000:2.0:1787784699.595950:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:2.0:1787784699.595962:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658026112:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1787784699.648902:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967330 is committed 00000001:00080000:2.0:1787784699.648925:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967329 is committed 00000100:00100000:1.0:1787784699.649176:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658026112:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 53181us (53387us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784699.649203:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:0.0:1787784699.654999:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.655142:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658026368 00000100:00100000:2.0:1787784699.655318:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658026368 00000100:00100000:2.0:1787784699.655460:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 777 00000100:00100000:2.0:1787784699.655476:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23297:x1874627658026368:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784699.655710:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784699.655748:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4f92 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a4f expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.655774:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784699.655780:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784699.655822:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.655840:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.655852:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784699.656295:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000048eea4a6 reply state 00000000b110d8a5 transno 4294967801 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.656307:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784699.656534:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23297:x1874627658026368:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 1060us (1550us total) trans 4294967801 rc 0/0 00000100:00100000:2.0:1787784699.656646:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 777 00000100:00100000:3.0:1787784699.659698:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:2.0:1787784699.659954:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.659971:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.659980:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.659987:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fda00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784699.659992:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.660001:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.660029:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784699.660033:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784699.660039:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.660046:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc4fa7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784699.660350:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658026496 00000100:00100000:0.0:1787784699.660364:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658026496 00000100:00100000:0.0:1787784699.660472:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:0.0:1787784699.660485:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658026496:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1787784699.660834:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:0.0:1787784699.660838:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb mtime from disk: 1787784698 -> 1536580263 00002000:00010000:0.0:1787784699.660841:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb atime from disk: 0 -> 1787784232 00002000:00010000:0.0:1787784699.660843:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb ctime from disk: 1787784698 -> 1787784698 00002000:00010000:0.0:1787784699.660846:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1787784699.660938:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658026496:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 455us (1250us total) trans 4294967331 rc 0/0 00000100:00100000:0.0:1787784699.660958:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:0.0:1787784699.663768:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.663872:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658026752 00000100:00100000:3.0:1787784699.663885:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658026752 00000100:00100000:3.0:1787784699.663970:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 778 00000100:00100000:3.0:1787784699.663981:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23297:x1874627658026752:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1787784699.664071:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.664093:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.664103:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 4/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.664196:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.664212:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 36 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.664229:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668369920:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784699.664346:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518ed69c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784699.667263:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.667367:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658026880 00000100:00100000:2.0:1787784699.667378:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658026880 00010000:00010000:2.0:1787784699.667435:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 37 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.667485:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 248 00000100:00100000:2.0:1787784699.667497:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2356:x1874627658026880:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784699.667516:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784699.667523:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 37 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.667537:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 37 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.667567:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784699.667573:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.667583:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.667592:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.667614:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784699.667678:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2356:x1874627658026880:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 182us (428us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.667687:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 248 00010000:00010000:3.0:1787784699.668128:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc4f76 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a41 expref: 36 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.668171:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668369920:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784699.668219:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668230:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668249:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668288:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668296:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668302:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668309:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.668313:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668321:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668334:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:3.0:1787784699.668337:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787784699.668342:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 2/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.668348:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fae lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.668405:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+35:23297:x1874627658026752:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 4425us (4652us total) trans 0 rc -22/-22 00000100:00100000:3.0:1787784699.668414:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 778 00000100:00100000:0.0:1787784699.671035:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.671280:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658027008 00000100:00100000:2.0:1787784699.671291:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658027008 00000100:00100000:2.0:1787784699.671353:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:2.0:1787784699.671365:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658027008:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784699.671426:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.671454:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.671465:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784699.671470:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784699.671475:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 4/1,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.671519:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.671536:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 10 pid: 13825 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.671554:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ee1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668370048:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784699.671728:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cf3000 going to sleep for 11 seconds 00000100:00100000:0.0:1787784699.675330:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784699.675692:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ee1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668370048:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784699.675725:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784699.676392:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658027136 00000100:00100000:1.0:1787784699.676468:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658027136 00010000:00010000:1.0:1787784699.676519:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 11 pid: 13825 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784699.676611:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 249 00000100:00100000:1.0:1787784699.676623:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658027136:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784699.676655:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784699.676685:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 11 pid: 13825 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.676701:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 11 pid: 13825 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.676731:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d5400fa/0xb0ce63137acc4fa0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a5d expref: 11 pid: 13825 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.676753:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784699.676758:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784699.676765:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.676798:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.676834:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784699.676913:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658027136:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 290us (1594us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784699.676921:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 249 00010000:00010000:2.0:1787784699.676984:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.676995:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.677024:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.677034:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.677043:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.677050:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fda00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784699.677054:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.677063:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.677077:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784699.677098:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc4fb5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.677177:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658027008:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 5815us (6156us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784699.677206:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:0.0:1787784699.680372:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.680488:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658027392 00000100:00100000:3.0:1787784699.680501:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658027392 00000100:00100000:3.0:1787784699.680579:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 779 00000100:00100000:3.0:1787784699.680591:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:23297:x1874627658027392:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784699.680800:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 3/0,1 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.680825:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.680839:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.681492:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.681509:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.681518:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.681621:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.681638:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 36 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.681654:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668370176:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784699.681827:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518ed69c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784699.685817:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.685932:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668370176:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784699.685967:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784699.686055:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658027520 00000100:00100000:2.0:1787784699.686064:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658027520 00010000:00010000:2.0:1787784699.686112:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 37 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784699.686220:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 250 00000100:00100000:1.0:1787784699.686238:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2356:x1874627658027520:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784699.686262:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784699.686271:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 37 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.686305:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 37 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.686328:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc4f84 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a48 expref: 37 pid: 5811 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.686356:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:1.0:1787784699.686360:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.686368:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.686374:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.686396:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784699.686468:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+36:2356:x1874627658027520:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 236us (666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784699.686477:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 250 00010000:00010000:3.0:1787784699.686945:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.686955:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784699.687395:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967802 00000001:00080000:3.0:1787784699.687598:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967803 00000001:00080000:3.0:1787784699.687647:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967804 00000100:00100000:3.0:1787784699.687751:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668370304:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787784699.687836:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784699.687882:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1a9180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784699.688093:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668370304 00000100:00100000:1.0:1787784699.688101:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668370304 00000100:00100000:1.0:1787784699.688219:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 25 00000100:00100000:1.0:1787784699.688230:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668370304:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787784699.688327:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668370304:12345-0@lo:1000:mdt00_001.0 Request processed in 99us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784699.688335:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 25 00000100:00100000:3.0:1787784699.688486:0:5812:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967330 gen 2 00000004:00080000:3.0:1787784699.688494:0:5812:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9894011ecb40, transno 4294967330 00000100:00100000:3.0:1787784699.688842:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668370304:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1787784699.690588:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967805 00000100:00100000:3.0:1787784699.690693:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668370432:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787784699.691042:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784699.691382:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784699.691710:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668370432 00000100:00100000:1.0:1787784699.691720:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668370432 00000100:00100000:0.0:1787784699.691826:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 26 00000100:00100000:0.0:1787784699.691841:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668370432:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787784699.692197:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5812:x1874627668370432:12345-0@lo:1000:mdt00_001.0 Request processed in 345us (1411us total) trans 4294967332 rc 0/0 00000100:00100000:0.0:1787784699.692207:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 26 00000100:00100000:3.0:1787784699.692523:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668370432:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1787784699.692580:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692591:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692621:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692627:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.692632:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692640:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692658:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989530cb1000) 00010000:00010000:3.0:1787784699.692662:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784699.692667:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692673:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fc3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692712:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692719:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692724:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692730:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fde00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.692733:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692750:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692759:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:3.0:1787784699.692763:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784699.692766:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 2/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692783:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc4f1b lrc: 0/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692811:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692826:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692837:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692844:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692850:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692855:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692861:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.692865:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692878:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692887:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:3.0:1787784699.692889:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784699.692892:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 2/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692897:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fca lrc: 0/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692908:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692914:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692920:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692925:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.692929:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692934:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692941:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989503a6d800) 00010000:00010000:3.0:1787784699.692944:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784699.692946:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 2/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.692962:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fbc lrc: 0/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784699.693518:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784699.693526:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784699.695124:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:0.0:1787784699.695145:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784699.696245:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23297:x1874627658027392:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 15642us (15873us total) trans 4294967806 rc 0/0 00000100:00100000:3.0:1787784699.696263:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 779 00000020:00080000:3.0:1787784699.696271:0:5812:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1787784699.704639:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.704763:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658027776 00000100:00100000:3.0:1787784699.704777:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658027776 00000100:00100000:3.0:1787784699.704857:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 158 00000100:00100000:3.0:1787784699.704869:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23297:x1874627658027776:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1787784699.705745:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x14:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1787784699.705871:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967807 00010000:00010000:3.0:1787784699.705931:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.705953:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.705968:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.705979:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.705987:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.705995:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.706034:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787784699.706200:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1787784699.706206:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23297:x1874627658027776:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1336us (1578us total) trans 4294967808 rc 0/0 00010000:00010000:1.0:1787784699.706210:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.706216:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 158 00010000:00010000:1.0:1787784699.706217:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc17b3420) 00000020:00080000:3.0:1787784699.706222:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1787784699.706223:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.706231:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.706246:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff989505542400) 00010000:00010000:1.0:1787784699.707275:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787784699.707287:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.707296:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4fd1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784699.707324:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784699.707327:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784699.707329:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784699.825717:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784699.925683:0:18024:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784699.925698:0:18024:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784699.925744:0:18024:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 19 count 2 00000040:00080000:1.0:1787784699.925751:0:18024:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784699.989818:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.990199:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658028032 00000100:00100000:3.0:1787784699.990213:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658028032 00000100:00100000:3.0:1787784699.990288:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 780 00000100:00100000:3.0:1787784699.990300:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23300:x1874627658028032:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784699.990337:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784699.990367:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fd8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a64 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.990475:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.990497:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.990512:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.990530:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.990599:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.990609:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991094:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4fed lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991106:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4fed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991114:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4fed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784699.991126:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:3.0:1787784699.991288:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991296:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991304:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991310:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.991315:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991324:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991339:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:3.0:1787784699.991342:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784699.991348:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991365:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc4fdf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991380:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991386:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991391:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991397:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.991400:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991406:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991414:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff989505542e00) 00010000:00010000:3.0:1787784699.991416:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784699.991420:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991426:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc4fe6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991481:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc4fd8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a64 expref: 44 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991510:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2400/0xb0ce63137acc4fed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a64 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.991518:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989505542400) 00010000:00010000:3.0:1787784699.991520:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784699.991523:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2400, rc 0) 00000100:00100000:3.0:1787784699.991620:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23300:x1874627658028032:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1321us (1828us total) trans 4294967809 rc 0/0 00000100:00100000:3.0:1787784699.991630:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 780 00000100:00100000:1.0:1787784699.998389:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784699.998472:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658028160 00000100:00100000:3.0:1787784699.998486:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658028160 00000100:00100000:3.0:1787784699.999179:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 781 00000100:00100000:3.0:1787784699.999209:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23300:x1874627658028160:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:3.0:1787784699.999307:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999327:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999341:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999377:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999385:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999405:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999411:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784699.999417:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999426:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999440:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989505542400) 00010000:00010000:3.0:1787784699.999443:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784699.999449:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784699.999456:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc4ffb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784699.999564:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23300:x1874627658028160:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 363us (1189us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784699.999572:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 781 00000100:00100000:0.0:1787784700.008434:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784700.009200:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658028416 00000100:00100000:3.0:1787784700.009216:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658028416 00000100:00100000:3.0:1787784700.009314:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 159 00000100:00100000:3.0:1787784700.009339:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23300:x1874627658028416:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:3.0:1787784700.009644:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:23300:x1874627658028416:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 319us (1222us total) trans 4294967810 rc 0/0 00000100:00100000:3.0:1787784700.009653:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 159 00000100:00080000:1.0:1787784700.010365:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784700.138461:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784700.138541:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658028544 00000100:00100000:2.0:1787784700.138554:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658028544 00000100:00100000:2.0:1787784700.138647:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 782 00000100:00100000:2.0:1787784700.138658:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+74:23316:x1874627658028544:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:2.0:1787784700.138691:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784700.138718:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5002 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a6b expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.138815:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.138833:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.138848:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.138883:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.138891:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.138897:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139452:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fcc00/0xb0ce63137acc5017 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139464:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fcc00/0xb0ce63137acc5017 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139472:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fcc00/0xb0ce63137acc5017 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784700.139558:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 7680 00000004:00200000:2.0:1787784700.139563:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784700.139647:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fcc00/0xb0ce63137acc5017 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139663:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139670:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139677:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139700:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784700.139705:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139714:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139731:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784700.139734:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784700.139739:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139745:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5009 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139760:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139766:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784700.139769:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784700.139770:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139775:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784700.139777:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139783:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139802:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff989530cb1100) 00010000:00010000:2.0:1787784700.139804:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784700.139807:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139811:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5010 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139845:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5002 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a6b expref: 76 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139856:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fcc00/0xb0ce63137acc5017 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a6b expref: 75 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.139861:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:2.0:1787784700.139863:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784700.139865:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fcc00, rc 0) 00000100:00100000:2.0:1787784700.139941:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:23316:x1874627658028544:12345-192.168.203.43@tcp:101:diff.0 Request processed in 1286us (1498us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784700.139948:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 782 00000100:00100000:3.0:1787784700.146025:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784700.146277:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658028672 00000100:00100000:1.0:1787784700.146310:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658028672 00000100:00100000:1.0:1787784700.146413:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 783 00000100:00100000:1.0:1787784700.146426:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+75:23316:x1874627658028672:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:1.0:1787784700.146455:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784700.146493:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc501e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a72 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.146584:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.146601:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.146615:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.146632:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.146639:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.146646:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147480:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc5033 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147494:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc5033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147503:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc5033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784700.147556:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00000004:00200000:1.0:1787784700.147561:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784700.147589:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc5033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147716:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc5033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147727:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147735:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147742:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147747:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784700.147752:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147760:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147840:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff9895029d1e00) 00010000:00010000:1.0:1787784700.147844:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784700.147850:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147857:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640000/0xb0ce63137acc5025 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147882:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147888:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147893:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147898:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784700.147901:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147907:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147914:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff9895029d1c00) 00010000:00010000:1.0:1787784700.147917:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784700.147919:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147925:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc502c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147964:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641000/0xb0ce63137acc501e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a72 expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147978:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640800/0xb0ce63137acc5033 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a72 expref: 76 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.147996:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989505542400) 00010000:00010000:1.0:1787784700.147998:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784700.148001:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640800, rc 0) 00000100:00100000:1.0:1787784700.148091:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23316:x1874627658028672:12345-192.168.203.43@tcp:101:diff.0 Request processed in 1666us (2080us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784700.148101:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 783 00000100:00100000:0.0:1787784700.154555:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784700.154676:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658028800 00000100:00100000:1.0:1787784700.154689:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658028800 00000100:00100000:1.0:1787784700.154867:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:1.0:1787784700.154881:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658028800:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:1.0:1787784700.154904:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784700.154929:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dc13da7/0xb0ce63137acc503a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a79 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784700.154963:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xf:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6a8f6c28, ctime: 0x6a8f6dfa, blocks: 0xb 00010000:00010000:1.0:1787784700.154989:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1787784700.154994:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00000100:00080000:2.0:1787784700.155000:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787784700.155008:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dc13da7/0xb0ce63137acc503a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a79 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.155033:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003dc13da7/0xb0ce63137acc503a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a79 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784700.155049:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641000, rc 0) 00000100:00100000:1.0:1787784700.155169:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658028800:12345-192.168.203.43@tcp:101:diff.0 Request processed in 270us (607us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784700.155177:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:3.0:1787784700.159517:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784700.160024:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658029056 00000100:00100000:2.0:1787784700.160039:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658029056 00000100:00100000:2.0:1787784700.160218:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 784 00000100:00100000:2.0:1787784700.160232:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23316:x1874627658029056:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:2.0:1787784700.160261:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784700.160293:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5041 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a80 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.160446:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5048 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.160465:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5048 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.160478:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5048 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784700.160491:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:2.0:1787784700.160782:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5041 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a80 expref: 79 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.160807:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5048 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a80 expref: 78 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784700.160817:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989505542400) 00010000:00010000:2.0:1787784700.160821:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784700.160824:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784700.160912:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23316:x1874627658029056:12345-192.168.203.43@tcp:101:diff.0 Request processed in 683us (1410us total) trans 4294967811 rc 0/0 00000100:00100000:2.0:1787784700.160922:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 784 00000100:00100000:0.0:1787784700.165942:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784700.166004:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784700.166089:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658029184 00000100:00100000:2.0:1787784700.166104:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658029184 00002000:00100000:2.0:1787784700.166231:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xf:0x0] (0->4095) req@ffff989539377c00 x1874627658029184/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:581/0 lens 488/0 e 0 to 0 dl 1787784711 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784700.166244:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539377c00 with x1874627658029184 ext(0->4095) 00002000:00010000:2.0:1787784700.166270:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989539377c00 00000100:00100000:2.0:1787784700.166301:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 62 00000100:00100000:2.0:1787784700.166313:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658029184:12345-192.168.203.43@tcp:3:diff.0 00000100:00100000:2.0:1787784700.166603:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658029184:12345-192.168.203.43@tcp:3:diff.0 Request processed in 290us (674us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784700.166613:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 62 00002000:00100000:2.0:1787784700.166621:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xf:0x0] (0->4095) req@ffff989539377c00 x1874627658029184/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:581/0 lens 488/4536 e 0 to 0 dl 1787784711 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784700.166630:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539377c00 with x1874627658029184 ext(0->4095) 00002000:00010000:2.0:1787784700.166634:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989539377c00 00000100:00100000:3.0:1787784700.166765:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658029312 00000100:00100000:3.0:1787784700.166770:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658029312 00000100:00100000:3.0:1787784700.166821:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 32 00000100:00100000:3.0:1787784700.166829:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:2357:x1874627658029312:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784700.166843:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784700.166876:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5048 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a80 expref: 78 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784700.166898:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff989505542400) 00010000:00010000:3.0:1787784700.166905:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784700.166944:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:2357:x1874627658029312:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 116us (940us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784700.166952:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 32 00000100:00100000:0.0:1787784700.175430:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784700.175536:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658029568 00000100:00100000:3.0:1787784700.175549:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658029568 00000100:00100000:3.0:1787784700.175623:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 160 00000100:00100000:3.0:1787784700.175635:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23316:x1874627658029568:12345-192.168.203.43@tcp:35:diff.0 00000001:00080000:3.0:1787784700.176016:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967812 00000100:00100000:3.0:1787784700.176263:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:23316:x1874627658029568:12345-192.168.203.43@tcp:35:diff.0 Request processed in 628us (848us total) trans 4294967813 rc 0/0 00000100:00100000:3.0:1787784700.176274:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 160 00000020:00080000:3.0:1787784700.176279:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1787784700.403915:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784700.403948:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668370560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784700.404047:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784700.404792:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668370560 00000100:00100000:1.0:1787784700.404806:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668370560 00000100:00100000:1.0:1787784700.404893:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:1.0:1787784700.404905:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668370560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784700.404936:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 64684032 pending 0 free 3857711104 avail 3854565376 00000100:00100000:1.0:1787784700.405022:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668370560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 119us (974us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784700.405030:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:0.0:1787784700.405088:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668370560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784700.694329:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784700.694374:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784700.694405:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784700.694428:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784700.694447:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784700.694464:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784700.694488:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784700.694528:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658029696 00000100:00100000:0.0:1787784700.694528:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658029952 00000100:00100000:2.0:1787784700.694541:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658029696 00000100:00100000:0.0:1787784700.694541:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658029952 00000100:00100000:0.0:1787784700.694612:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030464 00000100:00100000:0.0:1787784700.694615:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030464 00000100:00100000:2.0:1787784700.694619:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 527 00000100:00100000:2.0:1787784700.694634:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658029696:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784700.694646:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030080 00000100:00100000:0.0:1787784700.694650:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030080 00000100:00100000:2.0:1787784700.694728:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658029696:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 99us (415us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784700.694737:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 527 00000100:00100000:2.0:1787784700.694779:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030336 00000100:00100000:2.0:1787784700.694783:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030336 00000100:00100000:2.0:1787784700.694802:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 257 00000100:00100000:2.0:1787784700.694809:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658029952:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784700.694840:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658029952:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784700.694847:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 257 00000100:00100000:2.0:1787784700.694888:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 258 00000100:00100000:2.0:1787784700.694895:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658030464:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784700.694925:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658030464:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (496us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784700.694932:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 258 00000100:00100000:0.0:1787784700.694952:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 259 00000100:00100000:0.0:1787784700.694963:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658030336:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784700.695007:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658030336:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 46us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784700.695024:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030208 00000100:00100000:1.0:1787784700.695035:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030208 00000100:00100000:1.0:1787784700.695070:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658029824 00000100:00100000:1.0:1787784700.695073:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658029824 00000100:00100000:1.0:1787784700.695093:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00100000:1.0:1787784700.695099:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658030208:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784700.695136:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658030208:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (730us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784700.695142:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00100000:1.0:1787784700.695200:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 311 00000100:00100000:1.0:1787784700.695206:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:2357:x1874627658029824:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784700.695229:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:2357:x1874627658029824:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (781us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784700.695234:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 311 00000100:00100000:2.0:1787784700.695964:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 260 00000100:00100000:2.0:1787784700.695980:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658030080:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784700.696025:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2357:x1874627658030080:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 50us (1559us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784700.696031:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 260 00000100:00080000:2.0:1787784700.696070:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784700.704158:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 259 00010000:00010000:1.0:1787784700.872571:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784701.873604:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784702.450551:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895088512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668370944:0@lo:400:kworker.0 00000100:00100000:0.0:1787784702.450616:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784702.450673:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668371456:0@lo:400:kworker.0 00000100:00100000:0.0:1787784702.450695:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784702.450718:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784702.450723:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895088503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668371072:0@lo:400:kworker.0 00000100:00100000:0.0:1787784702.450740:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784702.450866:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668370688:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784702.450899:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784702.450921:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895088521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668371200:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.450939:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784702.450960:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784702.450965:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508850b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668370816:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.450988:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784702.451004:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668371328:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451020:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784702.451049:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668371456 00000100:00100000:2.0:1787784702.451055:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668371456 00000100:00100000:2.0:1787784702.451094:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668371200 00000100:00100000:2.0:1787784702.451096:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668371200 00000100:00100000:2.0:1787784702.451113:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1787784702.451129:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668371456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451217:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668371456:12345-0@lo:400:kworker.0 Request processed in 88us (520us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784702.451224:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1787784702.451240:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1787784702.451246:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668371200:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451280:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668371200:12345-0@lo:400:kworker.0 Request processed in 34us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784702.451286:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1787784702.451306:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668370944 00000100:00100000:2.0:1787784702.451310:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668370944 00000100:00100000:2.0:1787784702.451330:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668371072 00000100:00100000:2.0:1787784702.451333:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668371072 00000100:00100000:2.0:1787784702.451344:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668371328 00000100:00100000:2.0:1787784702.451347:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668371328 00000100:00100000:2.0:1787784702.451360:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:2.0:1787784702.451365:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668370944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451407:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668370944:12345-0@lo:400:kworker.0 Request processed in 41us (791us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784702.451413:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:2.0:1787784702.451426:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:2.0:1787784702.451432:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668371072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451473:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668371072:12345-0@lo:400:kworker.0 Request processed in 41us (731us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784702.451478:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:2.0:1787784702.451487:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1787784702.451492:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668371328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451524:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668371328:12345-0@lo:400:kworker.0 Request processed in 32us (502us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784702.451530:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00080000:2.0:1787784702.451545:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784702.451550:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784702.451552:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784702.451588:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668371328:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451599:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895088521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668371200:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451628:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895088503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668371072:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451637:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895088512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668370944:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.451645:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668371456:0@lo:400:kworker.0 00000100:00100000:0.0:1787784702.452055:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668370688 00000100:00100000:0.0:1787784702.452060:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668370688 00000100:00100000:0.0:1787784702.452106:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:0.0:1787784702.452112:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536d3dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668370688:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784702.452139:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784702.452217:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536d3dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668370688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 105us (1317us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784702.452222:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:0.0:1787784702.452408:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668370816 00000100:00100000:0.0:1787784702.452412:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668370816 00000100:00100000:0.0:1787784702.452432:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 528 00000100:00100000:0.0:1787784702.452437:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668370816:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784702.452617:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668370816:12345-0@lo:400:kworker.0 Request processed in 179us (1627us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784702.452624:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 528 00000100:00100000:2.0:1787784702.453901:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508850b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668370816:0@lo:400:kworker.0 00000100:00100000:2.0:1787784702.453930:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668370688:0@lo:13:osp-pre-1-0.0 00000040:00080000:1.0:1787784702.769576:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784702.769585:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784702.769621:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 19 count 2 00000040:00080000:1.0:1787784702.769629:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784702.901893:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784703.922342:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784704.945477:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784704.969471:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784704.969561:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030592 00000100:00100000:0.0:1787784704.969571:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030592 00000100:00100000:0.0:1787784704.969643:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 785 00000100:00100000:0.0:1787784704.969654:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:23232:x1874627658030592:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1787784704.969755:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:23232:x1874627658030592:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 104us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784704.969764:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 785 00000100:00080000:2.0:1787784704.972368:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1787784705.090339:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967811 is committed 00000001:00080000:2.0:1787784705.090357:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967801 is committed 00000001:00080000:2.0:1787784705.090363:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967813 is committed 00000001:00080000:2.0:1787784705.090372:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967810 is committed 00000020:00080000:2.0:1787784705.090390:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9895063ef180 refcount 2 committed 0 result 0 batchid 11 00000020:00080000:2.0:1787784705.090395:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895012e9c00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9895063efa80 00000020:00080000:2.0:1787784705.090398:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895012e9a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895401a0000 00000020:00080000:3.0:1787784705.090537:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 11 committed batchid 10 00000020:00080000:3.0:1787784705.090707:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 10 00000001:00080000:2.0:1787784705.093057:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967332 is committed 00000001:00080000:2.0:1787784705.093086:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967331 is committed 00000100:00100000:2.0:1787784705.527826:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668371584:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784705.527900:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784705.528116:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668371584 00000100:00100000:0.0:1787784705.528123:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668371584 00000100:00100000:0.0:1787784705.528225:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1787784705.528234:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668371584:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784705.528269:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 64684032 pending 0 free 3857711104 avail 3855613952 00000100:00080000:3.0:1787784705.529108:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784705.529368:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668371584:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1133us (1465us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784705.529377:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1787784705.529442:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967332 gen 2 00000004:00080000:2.0:1787784705.529448:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967332 committed transno 4294967332 00000100:00100000:2.0:1787784705.529472:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668371584:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784705.799312:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784705.799355:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784705.799388:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784705.799416:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784705.799444:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784705.799464:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784705.799491:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784705.799517:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030976 00000100:00100000:0.0:1787784705.799526:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030976 00000100:00100000:0.0:1787784705.799580:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:0.0:1787784705.799590:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658030976:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784705.799683:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658030976:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 93us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784705.799691:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:0.0:1787784705.799712:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658031488 00000100:00100000:0.0:1787784705.799716:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658031488 00000100:00100000:0.0:1787784705.799731:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658031104 00000100:00100000:0.0:1787784705.799733:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658031104 00000100:00100000:0.0:1787784705.799744:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658031360 00000100:00100000:0.0:1787784705.799746:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658031360 00000100:00100000:0.0:1787784705.799758:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:0.0:1787784705.799764:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658031488:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784705.799789:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658031488:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784705.799823:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:0.0:1787784705.799836:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 265 00000100:00100000:0.0:1787784705.799850:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658031104:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784705.799875:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:2356:x1874627658031104:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784705.799882:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 265 00000100:00100000:0.0:1787784705.799894:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 266 00000100:00100000:0.0:1787784705.799901:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658031360:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784705.799925:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658031360:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784705.799931:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 266 00000100:00100000:0.0:1787784705.799948:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658031232 00000100:00100000:0.0:1787784705.799952:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658031232 00000100:00100000:0.0:1787784705.799974:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030848 00000100:00100000:0.0:1787784705.799977:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030848 00000100:00100000:0.0:1787784705.799995:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:0.0:1787784705.800001:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2356:x1874627658031232:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784705.800038:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2356:x1874627658031232:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (638us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784705.800045:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:0.0:1787784705.800056:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 316 00000100:00100000:0.0:1787784705.800062:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:2356:x1874627658030848:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784705.800083:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:2356:x1874627658030848:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (639us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784705.800097:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 316 00000100:00080000:0.0:1787784705.800121:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784705.800125:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784705.800127:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784705.800704:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658030720 00000100:00100000:2.0:1787784705.800711:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658030720 00000100:00100000:2.0:1787784705.800752:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 529 00000100:00100000:2.0:1787784705.800759:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658030720:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784705.800811:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658030720:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 52us (1509us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784705.800818:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 529 00010000:00010000:0.0:1787784705.983972:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787784706.845355:0:18224:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 18:51:44 (1787784704) 00010000:00010000:2.0:1787784706.996929:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784707.140203:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784707.140486:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658031616 00000100:00100000:1.0:1787784707.140495:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658031616 00000100:00100000:1.0:1787784707.140618:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 786 00000100:00100000:1.0:1787784707.140638:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23931:x1874627658031616:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1787784707.140741:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23931:x1874627658031616:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 113us (547us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784707.140750:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 786 00000100:00080000:0.0:1787784707.142456:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784707.168801:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.168964:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658031744 00000100:00100000:0.0:1787784707.168978:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658031744 00000100:00100000:0.0:1787784707.169058:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 787 00000100:00100000:0.0:1787784707.169068:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23932:x1874627658031744:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784707.169091:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.169115:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5056 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a87 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169228:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169241:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169251:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169264:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169270:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169293:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784707.169506:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784707.169777:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169785:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169791:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169796:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.169800:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169810:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169821:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:0.0:1787784707.169824:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.169828:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169834:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc505d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169845:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169851:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169864:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169869:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.169871:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169878:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169886:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff989503a6d800) 00010000:00010000:0.0:1787784707.169888:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.169894:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169900:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5064 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169980:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5056 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a87 expref: 77 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.169991:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:0.0:1787784707.169993:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.169996:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5056 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a87 expref: 77 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.170005:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd600, rc 0) 00000100:00100000:0.0:1787784707.170107:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23932:x1874627658031744:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1041us (1314us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784707.170123:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 787 00000100:00100000:2.0:1787784707.195579:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784707.196017:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658032000 00000100:00100000:1.0:1787784707.196028:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658032000 00000100:00100000:1.0:1787784707.196084:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 788 00000100:00100000:1.0:1787784707.196095:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23932:x1874627658032000:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:1.0:1787784707.196211:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784707.196238:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc4e49 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b09f4 expref: 76 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.196259:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784707.196264:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784707.196530:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.196555:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.196566:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.196599:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 33 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.196624:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 33 pid: 10850 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784707.196640:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668371712:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784707.196747:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051dcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784707.204940:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668371712:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784707.204971:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784707.211766:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784707.212058:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658032128 00000100:00100000:1.0:1787784707.212068:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658032128 00010000:00010000:1.0:1787784707.212107:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 34 pid: 10850 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784707.212390:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 251 00000100:00100000:3.0:1787784707.212403:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2356:x1874627658032128:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784707.212418:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784707.212426:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 34 pid: 10850 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.212440:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 34 pid: 10850 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.212462:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc4e73 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a02 expref: 34 pid: 10850 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.212480:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:3.0:1787784707.212485:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.212494:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.212501:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.212544:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784707.212604:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+33:2356:x1874627658032128:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 204us (847us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784707.212612:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 251 00010000:00010000:0.0:1787784707.212689:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.212698:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.212731:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.212741:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.212750:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784707.214758:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 512 00010000:00010000:0.0:1787784707.214778:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.214788:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.214795:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.214801:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.214805:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.214812:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.214834:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989530cb1200) 00010000:00010000:0.0:1787784707.214836:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.214842:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.214847:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc506b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784707.215531:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000075127785 reply state 000000006412932e transno 4294967814 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784707.215542:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784707.215678:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23932:x1874627658032000:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 19580us (20103us total) trans 4294967814 rc 0/0 00000100:00100000:0.0:1787784707.215687:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 788 00010000:00010000:1.0:1787784707.218247:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.218263:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.218281:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.218286:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784707.218290:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.218298:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.218312:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff989503a6d800) 00010000:00010000:1.0:1787784707.218315:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784707.218320:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.218325:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5072 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784707.571207:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668371840:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784707.571350:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784707.571536:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668372224:0@lo:400:kworker.0 00000100:00100000:0.0:1787784707.571550:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668372096:0@lo:400:kworker.0 00000100:00100000:0.0:1787784707.571597:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784707.571651:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784707.571655:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668372608:0@lo:400:kworker.0 00000100:00100000:0.0:1787784707.571687:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784707.571725:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668372608 00000100:00100000:0.0:1787784707.571727:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668371968:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.571734:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668372608 00000100:00100000:0.0:1787784707.571760:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784707.571789:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668372480:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.571798:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1787784707.571810:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668372608:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784707.571818:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784707.571842:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784707.571849:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668372352:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.571879:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668372608:12345-0@lo:400:kworker.0 Request processed in 70us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.571884:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784707.571891:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1787784707.571911:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668372352 00000100:00100000:3.0:1787784707.571915:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668372352 00000100:00100000:0.0:1787784707.571935:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668372096 00000100:00100000:0.0:1787784707.571941:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668372096 00000100:00100000:3.0:1787784707.571945:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:3.0:1787784707.571954:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668372352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.571989:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668372352:12345-0@lo:400:kworker.0 Request processed in 38us (108us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784707.571996:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:3.0:1787784707.572013:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668372224 00000100:00100000:3.0:1787784707.572016:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668372224 00000100:00100000:2.0:1787784707.572064:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668371968 00000100:00100000:2.0:1787784707.572068:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668371968 00000100:00100000:2.0:1787784707.572093:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1787784707.572099:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668371968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784707.572140:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668371968:12345-0@lo:400:kworker.0 Request processed in 31us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784707.572166:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1787784707.572211:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668372480 00000100:00100000:2.0:1787784707.572220:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668372480 00000100:00100000:0.0:1787784707.573862:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1787784707.573882:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668372096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784707.573954:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668372096:12345-0@lo:400:kworker.0 Request processed in 81us (2356us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.573972:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:1.0:1787784707.574037:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668371840 00000100:00100000:1.0:1787784707.574045:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668371840 00000100:00100000:1.0:1787784707.574080:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:1.0:1787784707.574087:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668371840:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784707.574117:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784707.574184:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668371840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 98us (2835us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784707.574190:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00100000:3.0:1787784707.575814:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:3.0:1787784707.575830:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668372224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.575913:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668372224:12345-0@lo:400:kworker.0 Request processed in 88us (4275us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784707.575922:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00080000:1.0:1787784707.575947:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784707.575954:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784707.575957:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784707.575983:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668372224:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.576081:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668372096:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.576093:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668372608:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.576166:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668372352:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.576177:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668371968:0@lo:400:kworker.0 00000100:00100000:3.0:1787784707.576248:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668371840:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784707.576498:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:2.0:1787784707.576509:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668372480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784707.576555:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668372480:12345-0@lo:400:kworker.0 Request processed in 49us (4740us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784707.576561:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:2.0:1787784707.576590:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668372480:0@lo:400:kworker.0 00000100:00100000:2.0:1787784707.627509:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784707.627713:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658032256 00000100:00100000:3.0:1787784707.627723:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658032256 00000100:00100000:3.0:1787784707.628123:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 789 00000100:00100000:3.0:1787784707.628184:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23969:x1874627658032256:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:3.0:1787784707.628211:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784707.628235:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc5079 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a8e expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.628315:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5080 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.628329:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5080 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.628341:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5080 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784707.628399:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784707.628402:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 22 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784707.628426:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5080 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.628448:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc5079 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a8e expref: 78 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.628484:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5080 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a8e expref: 77 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.628493:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784707.628496:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784707.628499:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2a00, rc 0) 00000100:00100000:3.0:1787784707.628563:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23969:x1874627658032256:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 431us (1064us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784707.628570:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 789 00000100:00100000:1.0:1787784707.633896:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.634319:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658032512 00000100:00100000:0.0:1787784707.634329:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658032512 00000100:00100000:0.0:1787784707.634385:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 790 00000100:00100000:0.0:1787784707.634395:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23969:x1874627658032512:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:0.0:1787784707.634454:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784707.634479:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5080 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a8e expref: 77 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.634501:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542800) 00010000:00010000:0.0:1787784707.634510:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784707.634661:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23969:x1874627658032512:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 267us (773us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787784707.634669:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 790 00000100:00100000:2.0:1787784707.638454:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.638534:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658032640 00000100:00100000:0.0:1787784707.638543:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658032640 00000100:00100000:0.0:1787784707.638602:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 791 00000100:00100000:0.0:1787784707.638612:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+76:23969:x1874627658032640:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784707.638631:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.638653:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5087 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a95 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.638744:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc508e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.638757:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc508e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.638768:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc508e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784707.638821:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784707.638824:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 22 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784707.638843:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc508e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.638864:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5087 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a95 expref: 78 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.638880:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc508e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a95 expref: 77 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.638888:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784707.638891:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784707.638894:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd000, rc 0) 00000100:00100000:0.0:1787784707.639008:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23969:x1874627658032640:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 397us (562us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.639015:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 791 00000100:00100000:3.0:1787784707.646971:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.647228:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658032768 00000100:00100000:0.0:1787784707.647867:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658032768 00000100:00100000:0.0:1787784707.648082:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 792 00000100:00100000:0.0:1787784707.648094:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+77:23969:x1874627658032768:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784707.648115:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.648182:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc5095 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0a9c expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.648266:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.648287:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.648298:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.648311:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.648317:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.648322:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.648424:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc50aa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649014:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc50aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649026:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc50aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784707.649191:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 512 00000004:00200000:0.0:1787784707.649195:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784707.649222:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc50aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649236:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649534:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649541:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649548:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.649552:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649560:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649572:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784707.649575:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.649579:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649585:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc509c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649599:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649618:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649623:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649699:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.649704:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649711:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649721:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff989503a6df00) 00010000:00010000:0.0:1787784707.649724:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.649727:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649733:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50a3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649776:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc5095 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a9c expref: 79 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649791:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc50aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a9c expref: 78 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.649799:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784707.649801:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784707.649809:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16000, rc 0) 00000100:00100000:0.0:1787784707.649883:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23969:x1874627658032768:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 1791us (2965us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.649892:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 792 00000100:00100000:1.0:1787784707.742776:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.743190:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658032896 00000100:00100000:0.0:1787784707.743200:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658032896 00000100:00100000:0.0:1787784707.743260:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 793 00000100:00100000:0.0:1787784707.743271:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+33:23975:x1874627658032896:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784707.743292:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.743312:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50b1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0aa3 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.743396:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.743411:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.743422:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784707.743554:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784707.743559:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 22 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784707.743584:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.743610:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50b1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 35 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.743751:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 34 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.743768:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784707.743771:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784707.743775:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16400, rc 0) 00000100:00100000:0.0:1787784707.743866:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23975:x1874627658032896:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 597us (1098us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.743874:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 793 00000100:00100000:2.0:1787784707.763003:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784707.763193:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658033152 00000100:00100000:1.0:1787784707.763213:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658033152 00000100:00100000:1.0:1787784707.763270:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 794 00000100:00100000:1.0:1787784707.763280:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23975:x1874627658033152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784707.763299:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784707.763302:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787784707.763325:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc50aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a9c expref: 78 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.763344:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989503a6da00) 00010000:00010000:1.0:1787784707.763353:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784707.763367:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc50bf lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0aaa expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.763487:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.763499:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.763509:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.763520:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.763527:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.763550:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764372:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764385:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764393:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784707.764402:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 0 00010000:00010000:1.0:1787784707.764427:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764434:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764440:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764447:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784707.764450:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764458:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764467:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff98950a649900) 00010000:00010000:1.0:1787784707.764471:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784707.764482:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764489:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50c6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784707.764503:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c177eaee reply state 0000000004690481 transno 4294967815 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784707.764512:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787784707.764542:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc50bf lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 80 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764560:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 79 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.764567:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff98950a649100) 00010000:00010000:1.0:1787784707.764569:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784707.764571:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640400, rc 0) 00000100:00100000:1.0:1787784707.764981:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23975:x1874627658033152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1700us (1985us total) trans 4294967815 rc 0/0 00000100:00100000:1.0:1787784707.764990:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 794 00000100:00100000:2.0:1787784707.773042:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784707.773416:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658033408 00000100:00100000:3.0:1787784707.773427:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658033408 00000100:00100000:3.0:1787784707.773489:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 795 00000100:00100000:3.0:1787784707.773499:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23975:x1874627658033408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784707.773523:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784707.773545:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc50e2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ab1 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.773661:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.773677:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 81 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.773686:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.773734:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 81 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.773748:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 81 pid: 5812 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784707.773761:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668372736:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784707.773897:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503fa0300 going to sleep for 11 seconds 00010000:00010000:0.0:1787784707.774826:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.774839:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.774846:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.774852:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.774857:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.774865:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.774890:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4f0c1d45 (ffff98950a649300) 00010000:00010000:0.0:1787784707.774894:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.774898:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.774905:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc50cd lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784707.787791:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.787923:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658033536 00000100:00100000:0.0:1787784707.787931:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658033536 00010000:00010000:0.0:1787784707.787974:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 81 pid: 5812 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784707.788015:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 252 00000100:00100000:0.0:1787784707.788024:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:2357:x1874627658033536:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784707.788040:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784707.788046:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 81 pid: 5812 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.788060:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 81 pid: 5812 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.788083:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff98950a649100) 00010000:00010000:0.0:1787784707.788088:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.788097:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.788105:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.788125:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784707.788223:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:2357:x1874627658033536:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 199us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.788231:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 252 00010000:00010000:3.0:1787784707.788431:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc50d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aaa expref: 80 pid: 5812 timeout: 598 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784707.788449:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668372736:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784707.788473:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788482:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788487:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784707.788505:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 0 00010000:00010000:3.0:1787784707.788750:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788758:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788774:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788780:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784707.788784:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788792:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788802:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff98950a649100) 00010000:00010000:3.0:1787784707.788805:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784707.788808:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788815:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc50e9 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788867:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc50e2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ab1 expref: 80 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788877:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989505542d00) 00010000:00010000:3.0:1787784707.788880:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784707.788884:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc50e2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ab1 expref: 80 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.788897:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3a00, rc 0) 00000100:00100000:3.0:1787784707.788924:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:23975:x1874627658033408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 15427us (15891us total) trans 4294967816 rc 301/301 00000100:00100000:3.0:1787784707.788940:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 795 00000100:00100000:2.0:1787784707.795875:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784707.795962:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658033792 00000100:00100000:1.0:1787784707.795972:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658033792 00000100:00100000:1.0:1787784707.796030:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 161 00000100:00100000:1.0:1787784707.796039:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507678450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:23975:x1874627658033792:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784707.796562:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507678450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23975:x1874627658033792:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 522us (694us total) trans 4294967817 rc 0/0 00000100:00100000:1.0:1787784707.796587:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 161 00000100:00100000:1.0:1787784707.800355:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.800438:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658033920 00000100:00100000:0.0:1787784707.800448:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658033920 00000100:00100000:0.0:1787784707.800502:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 796 00000100:00100000:0.0:1787784707.800512:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23975:x1874627658033920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784707.800539:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.800561:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50f7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ab8 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.800624:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ab8 expref: 81 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.800714:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ab8 expref: 81 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.800743:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989503a6df00) 00010000:00010000:0.0:1787784707.800746:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784707.800750:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:0.0:1787784707.800807:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:23975:x1874627658033920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 296us (461us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.800815:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 796 00000100:00100000:2.0:1787784707.807048:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784707.807879:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658034176 00000100:00100000:1.0:1787784707.807890:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658034176 00000100:00100000:1.0:1787784707.808046:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 797 00000100:00100000:1.0:1787784707.808058:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:23975:x1874627658034176:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1787784707.808120:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 3/0,1 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808169:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808181:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808421:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808429:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808436:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808442:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784707.808446:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808453:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808605:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989503a6df00) 00010000:00010000:1.0:1787784707.808609:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787784707.808613:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 2/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.808619:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc50fe lrc: 0/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784707.808702:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:23975:x1874627658034176:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 645us (1661us total) trans 4294967818 rc 0/0 00000100:00100000:1.0:1787784707.808711:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 797 00000100:00100000:2.0:1787784707.811205:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.811880:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658034432 00000100:00100000:0.0:1787784707.811890:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658034432 00000100:00100000:0.0:1787784707.811953:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 798 00000100:00100000:0.0:1787784707.811964:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:23975:x1874627658034432:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784707.812788:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.812806:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.812833:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784707.812908:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967819 00000004:00010000:0.0:1787784707.812951:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008a96150f reply state 0000000084eec49c transno 4294967820 ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784707.812960:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784707.813106:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:23975:x1874627658034432:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 1142us (1905us total) trans 4294967820 rc 0/0 00000100:00100000:0.0:1787784707.813113:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 798 00000020:00080000:0.0:1787784707.813118:0:5813:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1787784707.815710:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.815736:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.815743:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.815748:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784707.815753:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.815760:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.815775:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989503a6df00) 00010000:00010000:1.0:1787784707.815778:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787784707.815782:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784707.815804:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc5105 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784707.817466:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784707.817540:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658034688 00000100:00100000:3.0:1787784707.817549:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658034688 00000100:00100000:3.0:1787784707.817604:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 162 00000100:00100000:3.0:1787784707.817614:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:23975:x1874627658034688:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784707.817875:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:23975:x1874627658034688:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 262us (416us total) trans 4294967821 rc 0/0 00000100:00100000:3.0:1787784707.817883:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 162 00000100:00100000:2.0:1787784707.867592:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784707.867747:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658034816 00000100:00100000:3.0:1787784707.867757:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658034816 00000100:00100000:3.0:1787784707.867812:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 799 00000100:00100000:3.0:1787784707.867822:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:23978:x1874627658034816:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784707.867847:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784707.867868:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc510c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0abf expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.867951:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.867966:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.867977:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.867990:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868008:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868015:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868117:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc5121 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868142:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc5121 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868183:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc5121 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784707.868214:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 512 00000004:00200000:3.0:1787784707.868217:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784707.868236:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc5121 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868247:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868263:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868270:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868276:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784707.868280:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868296:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868307:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:3.0:1787784707.868310:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784707.868314:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868320:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc5113 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868334:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868339:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868345:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868351:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784707.868353:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868360:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868365:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff989505542d00) 00010000:00010000:3.0:1787784707.868368:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784707.868371:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868385:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc511a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868413:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc510c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0abf expref: 87 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868425:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2e00/0xb0ce63137acc5121 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0abf expref: 86 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784707.868432:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989505542200) 00010000:00010000:3.0:1787784707.868434:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784707.868436:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2e00, rc 0) 00000100:00100000:3.0:1787784707.868492:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:23978:x1874627658034816:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 671us (910us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784707.868499:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 799 00000100:00100000:2.0:1787784707.872236:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784707.872447:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658035072 00000100:00100000:1.0:1787784707.872457:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658035072 00000100:00100000:1.0:1787784707.872549:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 800 00000100:00100000:1.0:1787784707.872559:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:23978:x1874627658035072:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784707.872580:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.872650:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc5128 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ac6 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872740:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872754:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872766:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872781:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872788:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872793:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872932:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc513d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872940:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc513d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.872946:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc513d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784707.872955:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:0.0:1787784707.873046:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873053:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873059:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873065:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.873069:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873077:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873089:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989505542200) 00010000:00010000:0.0:1787784707.873092:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.873096:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873114:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc512f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873482:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873489:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873494:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873499:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.873501:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873507:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873515:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4f0c1d45 (ffff989503a6de00) 00010000:00010000:0.0:1787784707.873517:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784707.873520:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873525:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5136 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873563:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc5128 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ac6 expref: 89 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873589:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc513d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ac6 expref: 88 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.873597:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989503a6df00) 00010000:00010000:0.0:1787784707.873600:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784707.873603:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbce00, rc 0) 00000100:00100000:0.0:1787784707.873668:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23978:x1874627658035072:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1111us (1442us total) trans 4294967822 rc 0/0 00000100:00100000:0.0:1787784707.873686:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 800 00000100:00100000:2.0:1787784707.877927:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784707.878063:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658035200 00000100:00100000:3.0:1787784707.878073:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658035200 00000100:00100000:1.0:1787784707.881247:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 801 00000100:00100000:0.0:1787784707.881460:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23978:x1874627658035200:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:0.0:1787784707.881538:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881555:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881566:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881592:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881599:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881605:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881627:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784707.881632:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881639:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881650:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989503a6df00) 00010000:00010000:0.0:1787784707.881653:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784707.881657:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.881663:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc514b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784707.881789:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23978:x1874627658035200:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 528us (3870us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.881798:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 801 00000100:00100000:1.0:1787784707.884700:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784707.884832:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658035456 00000100:00100000:3.0:1787784707.884840:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658035456 00000100:00100000:3.0:1787784707.884895:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 163 00000100:00100000:3.0:1787784707.884905:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23978:x1874627658035456:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:3.0:1787784707.885229:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:23978:x1874627658035456:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 325us (538us total) trans 4294967823 rc 0/0 00000100:00100000:3.0:1787784707.885236:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 163 00000040:00080000:1.0:1787784707.889354:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784707.889363:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784707.891038:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:1.0:1787784707.891055:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784707.937577:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.937666:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658035712 00000100:00100000:0.0:1787784707.937676:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658035712 00000100:00100000:0.0:1787784707.937735:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 802 00000100:00100000:0.0:1787784707.937745:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:23983:x1874627658035712:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784707.937768:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.937787:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5152 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0acd expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.937879:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5159 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.937909:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5159 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.937921:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5159 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784707.937932:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:0.0:1787784707.938046:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc5152 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0acd expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.938064:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5159 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0acd expref: 91 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.938073:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989503a6df00) 00010000:00010000:0.0:1787784707.938076:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1787784707.938079:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc400, rc 0) 00000100:00100000:0.0:1787784707.938302:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:23983:x1874627658035712:12345-192.168.203.43@tcp:101:cat.0 Request processed in 557us (734us total) trans 4294967824 rc 0/0 00000100:00100000:0.0:1787784707.938309:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 802 00000100:00100000:2.0:1787784707.944892:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784707.946033:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658035968 00000100:00100000:3.0:1787784707.946044:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658035968 00000100:00100000:2.0:1787784707.946386:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 33 00000100:00100000:2.0:1787784707.946396:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ef450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:23983:x1874627658035968:12345-192.168.203.43@tcp:103:cat.0 00010000:00010000:2.0:1787784707.946413:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787784707.946437:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc513d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ac6 expref: 91 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784707.946470:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5159 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0acd expref: 90 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784707.946493:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc50f7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ab8 expref: 89 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784707.946502:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989503a6df00) 00010000:00010000:2.0:1787784707.946513:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784707.946574:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ef450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23983:x1874627658035968:12345-192.168.203.43@tcp:103:cat.0 Request processed in 179us (1691us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784707.946581:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 33 00000100:00100000:1.0:1787784707.950297:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784707.952293:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658036096 00000100:00100000:2.0:1787784707.952306:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658036096 00000100:00100000:2.0:1787784707.952400:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 803 00000100:00100000:2.0:1787784707.952411:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23983:x1874627658036096:12345-192.168.203.43@tcp:58:cat.0 00010000:00010000:2.0:1787784707.952510:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784707.952527:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784707.952539:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784707.952737:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].19, 136 off10776 00000100:00100000:2.0:1787784707.952822:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23983:x1874627658036096:12345-192.168.203.43@tcp:58:cat.0 Request processed in 413us (2533us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784707.952844:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 803 00000100:00100000:3.0:1787784707.963619:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784707.963734:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658036224 00000100:00100000:0.0:1787784707.963745:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658036224 00000100:00100000:0.0:1787784707.963802:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 804 00000100:00100000:0.0:1787784707.963811:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:23983:x1874627658036224:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784707.963832:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784707.963850:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ad4 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.963925:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.963935:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 4/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ad4 expref: 89 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.963952:0:5812:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.963966:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 3/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ad4 expref: 89 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.963980:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:0.0:1787784707.963983:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784707.963987:0:5812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 3/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ad4 expref: 89 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784707.963995:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd600, rc 0) 00000100:00100000:0.0:1787784707.964062:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:23983:x1874627658036224:12345-192.168.203.43@tcp:101:cat.0 Request processed in 251us (452us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784707.964069:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 804 00010000:00010000:1.0:1787784708.017526:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784708.914143:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784708.914175:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784708.914235:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 20 count 3 00000040:00080000:0.0:1787784708.914243:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784708.914371:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668372864:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784708.914491:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503421cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784708.924182:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668372864:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784708.925248:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784708.925324:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658036352 00000100:00100000:0.0:1787784708.925331:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658036352 00000100:00100000:0.0:1787784708.925381:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 805 00000100:00100000:0.0:1787784708.925390:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23984:x1874627658036352:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784708.925420:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784708.925522:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23984:x1874627658036352:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 132us (277us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784708.925530:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 805 00000100:00100000:3.0:1787784708.930399:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784708.930501:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658036480 00000100:00100000:0.0:1787784708.930511:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658036480 00000100:00100000:0.0:1787784708.930653:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 806 00000100:00100000:0.0:1787784708.930664:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23984:x1874627658036480:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1787784708.930737:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930756:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930770:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930785:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930792:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930816:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930841:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930850:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930857:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784708.930895:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00000004:00200000:0.0:1787784708.930900:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784708.930911:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930922:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930928:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930935:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930941:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.930945:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930952:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930971:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784708.930975:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784708.930980:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.930987:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc5175 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931000:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931006:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931012:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931017:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.931020:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931026:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931033:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3d0032c8 (ffff989503a6de00) 00010000:00010000:0.0:1787784708.931046:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784708.931049:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931056:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc517c lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931081:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931088:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931094:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931099:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.931102:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931108:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931116:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:0.0:1787784708.931132:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784708.931136:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.931142:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5183 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784708.931260:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23984:x1874627658036480:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 597us (869us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784708.931268:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 806 00000100:00100000:3.0:1787784708.939844:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784708.939981:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658036736 00000100:00100000:1.0:1787784708.939990:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658036736 00000100:00100000:1.0:1787784708.940096:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 807 00000100:00100000:1.0:1787784708.940107:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:23984:x1874627658036736:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784708.940142:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784708.940183:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc518a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0adb expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940287:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940319:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940332:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784708.940355:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:1.0:1787784708.940452:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940460:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940468:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940474:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784708.940479:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940487:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940500:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:1.0:1787784708.940503:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784708.940507:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940514:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc5191 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940565:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc518a lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0adb expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940577:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784708.940579:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784708.940583:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc518a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0adb expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.940591:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784708.940652:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+35:23984:x1874627658036736:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 546us (816us total) trans 4294967825 rc 301/301 00000100:00100000:1.0:1787784708.940659:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 807 00000100:00100000:1.0:1787784708.954047:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784708.954354:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658036864 00000100:00100000:3.0:1787784708.954365:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658036864 00000100:00100000:3.0:1787784708.954456:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 808 00000100:00100000:3.0:1787784708.954468:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:23984:x1874627658036864:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784708.954505:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784708.954615:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:23984:x1874627658036864:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 150us (577us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784708.954623:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 808 00000100:00100000:3.0:1787784708.957280:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784708.957366:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658036992 00000100:00100000:0.0:1787784708.957377:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658036992 00000100:00100000:0.0:1787784708.957435:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 809 00000100:00100000:0.0:1787784708.957460:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:23984:x1874627658036992:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784708.957480:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784708.957500:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc519f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ae2 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957563:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957577:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957588:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957603:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957610:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957617:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957732:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc51b4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957742:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc51b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957751:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc51b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784708.957793:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 512 00000004:00200000:0.0:1787784708.957797:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784708.957819:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc51b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957831:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957837:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957844:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957850:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c17e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.957854:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957862:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957874:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784708.957876:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784708.957881:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957886:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc51a6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957899:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957914:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957920:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957924:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.957928:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957934:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957943:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff989503a6de00) 00010000:00010000:0.0:1787784708.957945:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784708.957948:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957954:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ad lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957983:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc519f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ae2 expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.957996:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17000/0xb0ce63137acc51b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ae2 expref: 36 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.958003:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989505542200) 00010000:00010000:0.0:1787784708.958005:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784708.958013:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c17000, rc 0) 00000100:00100000:0.0:1787784708.958078:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23984:x1874627658036992:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 620us (805us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784708.958085:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 809 00000100:00100000:2.0:1787784708.966443:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784708.966887:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658037248 00000100:00100000:1.0:1787784708.966897:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658037248 00000100:00100000:1.0:1787784708.967001:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 810 00000100:00100000:1.0:1787784708.967011:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:23984:x1874627658037248:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784708.967031:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784708.967051:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc51bb lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ae9 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787784708.968037:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x11:0x0], index 1: 0 00010000:00010000:1.0:1787784708.968549:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc51c2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.968566:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc51c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.968588:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc51c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.968656:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc51bb lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ae9 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.968677:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc51c2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ae9 expref: 38 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784708.968687:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff98950a649900) 00010000:00010000:1.0:1787784708.968691:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784708.968694:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641a00, rc 0) 00000100:00100000:1.0:1787784708.968779:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23984:x1874627658037248:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1769us (2345us total) trans 4294967826 rc 0/0 00000100:00100000:1.0:1787784708.968788:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 810 00000100:00100000:1.0:1787784708.974460:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784708.974552:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658037504 00000100:00100000:0.0:1787784708.974562:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658037504 00000100:00100000:0.0:1787784708.974620:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 811 00000100:00100000:0.0:1787784708.974636:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23984:x1874627658037504:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784708.974694:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784708.974721:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc51c2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ae9 expref: 38 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974741:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff98950a649900) 00010000:00010000:0.0:1787784708.974751:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784708.974773:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974787:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974796:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974811:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974819:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974825:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974907:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974915:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974922:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974929:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.974933:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974941:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974951:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989505542200) 00010000:00010000:0.0:1787784708.974954:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784708.974957:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974964:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc51d0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974974:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974980:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974987:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.974999:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.975002:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.975009:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.975018:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x3eeac0b8 (ffff989503a6de00) 00010000:00010000:0.0:1787784708.975020:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784708.975023:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.975029:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51d7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x3eeac0b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784708.975112:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23984:x1874627658037504:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 482us (659us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784708.975135:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 811 00000100:00100000:2.0:1787784708.980599:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784708.980689:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658037632 00000100:00100000:3.0:1787784708.980699:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658037632 00000100:00100000:3.0:1787784708.980756:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 812 00000100:00100000:3.0:1787784708.980768:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23984:x1874627658037632:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784708.980788:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784708.980811:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51de lrc: 2/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0af0 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.980845:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.980860:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.980871:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.980983:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51de lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 39 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.981001:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.981010:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784708.981014:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784708.981017:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:3.0:1787784708.981079:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23984:x1874627658037632:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 315us (489us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784708.981086:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 812 00000100:00100000:3.0:1787784708.986705:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784708.986810:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658037888 00000100:00100000:0.0:1787784708.986825:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658037888 00000100:00100000:0.0:1787784708.986885:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 813 00000100:00100000:0.0:1787784708.986897:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23984:x1874627658037888:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784708.986978:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.986995:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.987008:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784708.987371:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].9, 88 off8840 00000004:00080000:0.0:1787784708.987382:0:5812:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 500074234504ns, next at 501074231878ns, rc = 0 00000004:00010000:0.0:1787784708.987471:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000052edfbc3 reply state 000000004c25733e transno 4294967827 ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784708.987481:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784708.987578:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:23984:x1874627658037888:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 683us (881us total) trans 4294967827 rc 0/0 00000100:00100000:0.0:1787784708.987585:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 813 00010000:00010000:0.0:1787784708.990045:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.990060:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.990068:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.990074:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784708.990078:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.990087:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.990103:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:0.0:1787784708.990107:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784708.990111:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 2/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784708.990318:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc51ec lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784708.995051:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784708.995134:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658038016 00000100:00100000:3.0:1787784708.995144:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658038016 00000100:00100000:3.0:1787784708.995219:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 814 00000100:00100000:3.0:1787784708.995228:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23984:x1874627658038016:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784708.995260:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784708.995280:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0af7 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.995337:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 40 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.995351:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 40 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784708.995388:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784708.995390:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784708.995394:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3a00, rc 0) 00000100:00100000:3.0:1787784708.995441:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:23984:x1874627658038016:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 214us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784708.995448:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 814 00000100:00100000:2.0:1787784709.001539:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784709.001633:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658038144 00000100:00100000:1.0:1787784709.001643:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658038144 00000100:00100000:1.0:1787784709.001720:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 197 00000100:00100000:1.0:1787784709.001733:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658038144:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1787784709.002053:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658038144:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 322us (521us total) trans 4294967336 rc 0/0 00000100:00100000:1.0:1787784709.002062:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 197 00000100:00080000:2.0:1787784709.002749:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784709.011139:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.011251:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658038400 00000100:00100000:3.0:1787784709.011260:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658038400 00000100:00100000:3.0:1787784709.011329:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 164 00000100:00100000:3.0:1787784709.011339:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:23984:x1874627658038400:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784709.011555:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23984:x1874627658038400:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 217us (437us total) trans 4294967828 rc 0/0 00000100:00100000:3.0:1787784709.011563:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 164 00000100:00080000:2.0:1787784709.011717:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784709.013825:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784709.014016:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658038656 00000100:00100000:2.0:1787784709.014025:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658038656 00000100:00100000:2.0:1787784709.014082:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 815 00000100:00100000:2.0:1787784709.014092:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23984:x1874627658038656:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784709.014331:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23984:x1874627658038656:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 240us (513us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784709.014338:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 815 00000100:00100000:0.0:1787784709.016865:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.016947:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658038784 00000100:00100000:3.0:1787784709.016955:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658038784 00000100:00100000:3.0:1787784709.017011:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 816 00000100:00100000:3.0:1787784709.017021:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23984:x1874627658038784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784709.017040:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784709.017061:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc51fa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0afe expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.017184:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5201 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.017216:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5201 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.017228:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5201 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784709.017264:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784709.017283:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5201 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.017309:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc51fa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0afe expref: 43 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.017328:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5201 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0afe expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.017337:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784709.017339:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1787784709.017343:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2a00, rc 0) 00000100:00100000:3.0:1787784709.017406:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23984:x1874627658038784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 387us (550us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784709.017413:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 816 00000100:00100000:3.0:1787784709.022941:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.023030:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658038912 00000100:00100000:0.0:1787784709.023039:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658038912 00000100:00100000:0.0:1787784709.023092:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 198 00000100:00100000:0.0:1787784709.023102:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894040083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658038912:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784709.023134:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784709.023189:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b05 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784709.023216:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x11:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a8f6e04, blocks: 0x1 00010000:00010000:0.0:1787784709.023235:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=500s v=(0 0) 00010000:00010000:0.0:1787784709.023240:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784709.023249:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.023268:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.023281:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:0.0:1787784709.023532:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894040083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658038912:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 431us (600us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784709.023541:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 198 00000100:00100000:2.0:1787784709.026305:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.026423:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658039168 00000100:00100000:0.0:1787784709.026432:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658039168 00000100:00100000:0.0:1787784709.026527:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 817 00000100:00100000:0.0:1787784709.026538:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23984:x1874627658039168:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784709.026645:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23984:x1874627658039168:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 109us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784709.026653:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 817 00000100:00100000:2.0:1787784709.029589:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.029675:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658039296 00000100:00100000:3.0:1787784709.029683:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658039296 00000100:00100000:3.0:1787784709.029746:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 199 00000100:00100000:3.0:1787784709.029756:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305161c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:23984:x1874627658039296:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784709.029773:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784709.029807:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b0c expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.029837:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.029846:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784709.029850:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784709.029856:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 4/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.029889:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.029902:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 12 pid: 6670 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784709.029916:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668372992:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784709.030012:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503fa0900 going to sleep for 11 seconds 00000100:00100000:2.0:1787784709.033186:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.033359:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668372992:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784709.033397:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.033416:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3400, rc 0) 00000100:00100000:3.0:1787784709.033467:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305161c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:23984:x1874627658039296:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3712us (3885us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784709.033492:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 199 00000100:00100000:0.0:1787784709.033920:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658039424 00000100:00100000:0.0:1787784709.033928:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658039424 00010000:00010000:0.0:1787784709.033961:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 12 pid: 6670 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784709.033998:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 253 00000100:00100000:0.0:1787784709.034008:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658039424:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784709.034020:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784709.034041:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 12 pid: 6670 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.034053:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 12 pid: 6670 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.034076:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc5208 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b05 expref: 12 pid: 6670 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.034090:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784709.034094:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784709.034110:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.034135:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.034198:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784709.034212:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668373120:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784709.034274:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503421cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784709.038371:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668373120:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784709.038398:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784709.038453:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658039424:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 4447us (5307us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784709.038465:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 253 00010000:00010000:0.0:1787784709.041373:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784709.042610:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784709.042795:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658039552 00000100:00100000:2.0:1787784709.042805:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658039552 00002000:00100000:2.0:1787784709.042854:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x11:0x0] (0->157) req@ffff98940400bc00 x1874627658039552/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:590/0 lens 648/0 e 0 to 0 dl 1787784720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784709.042864:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940400bc00 with x1874627658039552 ext(0->157) 00002000:00010000:2.0:1787784709.042870:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98940400bc00 00000100:00100000:2.0:1787784709.042937:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 63 00000100:00100000:2.0:1787784709.042948:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658039552:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1787784709.043422:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658039552:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 474us (817us total) trans 4294967337 rc 0/0 00000100:00100000:2.0:1787784709.043431:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 63 00002000:00100000:2.0:1787784709.043438:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x11:0x0] (0->157) req@ffff98940400bc00 x1874627658039552/t4294967337(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:590/0 lens 648/448 e 0 to 0 dl 1787784720 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784709.043448:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940400bc00 with x1874627658039552 ext(0->157) 00002000:00010000:2.0:1787784709.043452:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98940400bc00 00000100:00100000:1.0:1787784709.047383:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.047509:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658039680 00000100:00100000:3.0:1787784709.047518:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658039680 00000100:00100000:3.0:1787784709.047570:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 818 00000100:00100000:3.0:1787784709.047581:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:23984:x1874627658039680:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1787784709.081189:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967827 is committed 00000004:00080000:2.0:1787784709.081203:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 10 committed 00000001:00080000:2.0:1787784709.081216:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967824 is committed 00000001:00080000:2.0:1787784709.081241:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967828 is committed 00000020:00080000:2.0:1787784709.081251:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 11 batchid updated 00000020:00080000:2.0:1787784709.081266:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 11 00000020:00080000:2.0:1787784709.081276:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 11 00000100:00100000:0.0:1787784709.082201:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784709.082221:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668373248:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1787784709.082289:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784709.082670:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23984:x1874627658039680:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 35088us (35295us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784709.082697:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 818 00000100:00100000:1.0:1787784709.082733:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668373248 00000100:00100000:1.0:1787784709.082740:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668373248 00000100:00100000:1.0:1787784709.083566:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1787784709.083576:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668373248:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1787784709.084342:0:10688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:1.0:1787784709.084348:0:10688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb mtime from disk: 0 -> 1787784708 00002000:00010000:1.0:1787784709.084350:0:10688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1787784709.084352:0:10688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb ctime from disk: 1787784708 -> 1787784708 00002000:00010000:1.0:1787784709.084355:0:10688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:1.0:1787784709.084415:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668373248:12345-0@lo:2:osp-syn-1-0.0 Request processed in 839us (2124us total) trans 4294967338 rc 0/0 00000100:00100000:1.0:1787784709.084423:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000004:00080000:1.0:1787784709.084475:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff98953f7b2580/2, rc 0, transno 4294967338 00000100:00100000:1.0:1787784709.084482:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668373248:0@lo:2:osp-syn-1-0.0 00000100:00080000:0.0:1787784709.085260:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784709.085590:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784709.085662:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658039808 00000100:00100000:1.0:1787784709.085669:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658039808 00000100:00100000:0.0:1787784709.085831:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 201 00000100:00100000:0.0:1787784709.085841:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658039808:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:1.0:1787784709.127109:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967338 is committed 00000001:00080000:1.0:1787784709.127226:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967336 is committed 00000001:00080000:1.0:1787784709.127243:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967337 is committed 00000100:00100000:1.0:1787784709.127494:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658039808:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 41645us (41897us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784709.127507:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 201 00000100:00100000:2.0:1787784709.137755:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.137860:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658040064 00000100:00100000:3.0:1787784709.137869:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658040064 00000100:00100000:3.0:1787784709.137927:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 819 00000100:00100000:3.0:1787784709.137937:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23984:x1874627658040064:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784709.137992:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784709.138018:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2a00/0xb0ce63137acc5201 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0afe expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.138037:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784709.138042:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784709.138068:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.138081:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.138090:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784709.139025:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000aba85a06 reply state 0000000088cb14e3 transno 4294967829 ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784709.139053:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784709.139176:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:23984:x1874627658040064:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 1238us (1427us total) trans 4294967829 rc 0/0 00000100:00100000:1.0:1787784709.139183:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 819 00010000:00010000:1.0:1787784709.142452:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.142471:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.142479:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.142484:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784709.142489:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.142496:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.142513:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:1.0:1787784709.142517:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784709.142522:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.142528:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc5216 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784709.144406:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.144484:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658040192 00000100:00100000:3.0:1787784709.144495:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658040192 00000100:00100000:3.0:1787784709.144557:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 202 00000100:00100000:3.0:1787784709.144567:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527dea450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658040192:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1787784709.144812:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:3.0:1787784709.144816:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb mtime from disk: 1787784708 -> 1536580263 00002000:00010000:3.0:1787784709.144819:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb atime from disk: 0 -> 1787784232 00002000:00010000:3.0:1787784709.144821:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb ctime from disk: 1787784708 -> 1787784707 00002000:00010000:3.0:1787784709.144824:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1787784709.144889:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527dea450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658040192:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 322us (491us total) trans 4294967339 rc 0/0 00000100:00100000:3.0:1787784709.144897:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 202 00000100:00100000:2.0:1787784709.152393:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784709.152489:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658040448 00000100:00100000:1.0:1787784709.152499:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658040448 00000100:00100000:1.0:1787784709.152576:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 820 00000100:00100000:1.0:1787784709.152587:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:23984:x1874627658040448:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1787784709.152654:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.152681:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.152691:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 4/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.152728:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.152742:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 38 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784709.152756:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668373376:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784709.152847:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784709.156875:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784709.157223:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658040576 00000100:00100000:2.0:1787784709.157232:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658040576 00010000:00010000:2.0:1787784709.157269:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 39 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784709.157304:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 254 00000100:00100000:2.0:1787784709.157313:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2356:x1874627658040576:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784709.157328:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784709.157335:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 39 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.157360:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 39 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.157382:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:2.0:1787784709.157388:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.157398:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.157405:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.157422:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784709.157478:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2356:x1874627658040576:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 165us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784709.157485:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 254 00010000:00010000:1.0:1787784709.157640:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc51e5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af0 expref: 38 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784709.157656:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668373376:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784709.157696:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157714:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157736:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157765:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157772:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157779:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157784:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784709.157788:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157795:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157804:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:1.0:1787784709.157807:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787784709.157810:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.157816:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc521d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784709.157855:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23984:x1874627658040448:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 5270us (5470us total) trans 0 rc -22/-22 00000100:00100000:1.0:1787784709.157864:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 820 00000100:00100000:2.0:1787784709.161536:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.161635:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658040704 00000100:00100000:3.0:1787784709.161645:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658040704 00000100:00100000:3.0:1787784709.161850:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 203 00000100:00100000:3.0:1787784709.161861:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658040704:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784709.161901:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.161922:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.161931:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784709.161936:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784709.161940:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.161972:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.161999:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 11 pid: 13825 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784709.162013:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668373504:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784709.162089:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503421d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784709.165005:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668373504:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784709.165036:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784709.167245:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784709.167565:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658040832 00000100:00100000:1.0:1787784709.167573:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658040832 00010000:00010000:1.0:1787784709.167614:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 12 pid: 13825 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784709.167648:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 255 00000100:00100000:1.0:1787784709.167656:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658040832:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784709.167683:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784709.167706:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 12 pid: 13825 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.167720:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 12 pid: 13825 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.167751:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b283eea7/0xb0ce63137acc520f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b0c expref: 12 pid: 13825 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.167768:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784709.167772:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784709.167778:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.167788:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.167819:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784709.167876:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658040832:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 220us (639us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784709.167897:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 255 00010000:00010000:3.0:1787784709.167962:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.167972:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.167997:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.168006:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.168013:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.168034:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784709.168038:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.168048:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.168060:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784709.168069:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd6d3335/0xb0ce63137acc5224 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784709.168114:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658040704:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 6255us (6588us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784709.168137:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 203 00000100:00100000:0.0:1787784709.173412:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784709.173560:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658041088 00000100:00100000:2.0:1787784709.173568:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658041088 00000100:00100000:2.0:1787784709.173626:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 821 00000100:00100000:2.0:1787784709.173636:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:23984:x1874627658041088:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784709.173726:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.173743:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.173765:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.173984:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.173995:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.174003:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.174033:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.174047:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 38 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784709.174060:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668373632:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784709.174183:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787784709.178236:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.178358:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668373632:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784709.178383:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784709.178644:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658041216 00000100:00100000:1.0:1787784709.178652:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658041216 00010000:00010000:1.0:1787784709.178902:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 39 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784709.178980:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 256 00000100:00100000:1.0:1787784709.178991:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2356:x1874627658041216:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784709.179005:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784709.179012:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 39 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.179023:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 39 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.179044:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc51f3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0af7 expref: 39 pid: 5813 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.179060:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:1.0:1787784709.179065:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.179074:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.179081:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.179136:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784709.179223:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2356:x1874627658041216:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 234us (996us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784709.179231:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 256 00010000:00010000:0.0:1787784709.179329:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.179336:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784709.179515:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967830 00000001:00080000:0.0:1787784709.179670:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967831 00000001:00080000:0.0:1787784709.179730:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967832 00000100:00100000:0.0:1787784709.179806:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668373760:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784709.179853:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784709.179886:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784709.179943:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668373760 00000100:00100000:2.0:1787784709.179948:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668373760 00000100:00100000:2.0:1787784709.179985:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 27 00000100:00100000:2.0:1787784709.179993:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668373760:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787784709.180055:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668373760:12345-0@lo:1000:mdt00_003.0 Request processed in 64us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784709.180073:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 27 00000100:00100000:0.0:1787784709.180280:0:10850:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967338 gen 2 00000004:00080000:0.0:1787784709.180286:0:10850:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff98953f7b2580, transno 4294967338 00000100:00100000:0.0:1787784709.180305:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668373760:0@lo:1000:mdt00_003.0 00000001:00080000:0.0:1787784709.180614:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967833 00000100:00100000:0.0:1787784709.180662:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668373888:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784709.180691:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784709.180731:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668373888 00000100:00100000:0.0:1787784709.180735:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668373888 00000100:00100000:0.0:1787784709.180763:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 28 00000100:00100000:0.0:1787784709.180770:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668373888:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784709.180901:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:10850:x1874627668373888:12345-0@lo:1000:mdt00_003.0 Request processed in 130us (207us total) trans 4294967340 rc 0/0 00000100:00100000:0.0:1787784709.180926:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 28 00000100:00100000:0.0:1787784709.180951:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784709.180969:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b0000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668373888:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787784709.180994:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181003:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181010:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181015:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784709.181019:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181026:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181038:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989505542600) 00010000:00010000:0.0:1787784709.181041:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784709.181045:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181050:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5232 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181076:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181082:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181094:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181099:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784709.181102:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181109:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181282:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:0.0:1787784709.181285:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784709.181291:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ad4 expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181300:0:10850:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ad4 expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181308:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ad4 expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.181331:0:10850:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd600/0xb0ce63137acc516e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ad4 expref: 77 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784709.181341:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668374016:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784709.181386:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180780 going to sleep for 11 seconds 00000100:00100000:3.0:1787784709.185466:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784709.185541:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658041344 00000100:00100000:1.0:1787784709.185552:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658041344 00000100:00100000:1.0:1787784709.185621:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00100000:1.0:1787784709.185632:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23983:x1874627658041344:12345-192.168.203.43@tcp:49:cat.0 00000100:00100000:1.0:1787784709.185773:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23983:x1874627658041344:12345-192.168.203.43@tcp:49:cat.0 Request processed in 143us (314us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784709.185781:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 165 00000100:00080000:2.0:1787784709.186685:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784709.188974:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784709.189103:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658041472 00000100:00100000:1.0:1787784709.189126:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658041472 00000100:00100000:1.0:1787784709.189210:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 204 00000100:00100000:1.0:1787784709.189219:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+10:23983:x1874627658041472:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784709.189236:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784709.189256:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d92b120f/0xb0ce63137acc5239 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b13 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784709.189279:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x11:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6a8f6c28, ctime: 0x6a8f6e03, blocks: 0xb 00010000:00010000:1.0:1787784709.189298:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=500s v=(0 0) 00010000:00010000:1.0:1787784709.189302:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784709.189311:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d92b120f/0xb0ce63137acc5239 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b13 expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.189346:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d92b120f/0xb0ce63137acc5239 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b13 expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784709.189358:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640000, rc 0) 00000100:00100000:1.0:1787784709.189412:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:23983:x1874627658041472:12345-192.168.203.43@tcp:101:cat.0 Request processed in 194us (446us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784709.189419:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 204 00000100:00080000:2.0:1787784709.189606:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784709.192535:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.192646:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668374016:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784709.192687:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192698:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5167 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192739:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192752:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192762:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192772:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192779:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192786:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192791:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784709.192795:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192802:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192811:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:0.0:1787784709.192817:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784709.192822:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192828:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5240 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192837:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192843:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192849:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192854:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784709.192857:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192863:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192870:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:0.0:1787784709.192872:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784709.192874:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.192880:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc522b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784709.193073:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23984:x1874627658041088:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 19438us (19669us total) trans 4294967834 rc 0/0 00000100:00100000:0.0:1787784709.193091:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 821 00000020:00080000:0.0:1787784709.193096:0:10850:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1787784709.193267:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784709.193272:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784709.193762:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1787784709.193771:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784709.193884:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658041600 00000100:00100000:3.0:1787784709.193893:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658041600 00002000:00100000:3.0:1787784709.193934:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x11:0x0] (0->4095) req@ffff989539102d00 x1874627658041600/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:590/0 lens 488/0 e 0 to 0 dl 1787784720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784709.193945:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539102d00 with x1874627658041600 ext(0->4095) 00002000:00010000:3.0:1787784709.193950:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539102d00 00000100:00100000:3.0:1787784709.193977:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 64 00000100:00100000:3.0:1787784709.193994:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658041600:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:3.0:1787784709.194203:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658041600:12345-192.168.203.43@tcp:3:cat.0 Request processed in 209us (1675us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784709.194210:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 64 00002000:00100000:3.0:1787784709.194217:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x11:0x0] (0->4095) req@ffff989539102d00 x1874627658041600/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:590/0 lens 488/4536 e 0 to 0 dl 1787784720 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784709.194225:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539102d00 with x1874627658041600 ext(0->4095) 00002000:00010000:3.0:1787784709.194228:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539102d00 00000100:00100000:1.0:1787784709.202262:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784709.202307:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.202396:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658041856 00000100:00100000:0.0:1787784709.202406:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658041856 00000100:00100000:0.0:1787784709.202445:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658042112 00000100:00100000:0.0:1787784709.202448:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658042112 00000100:00100000:0.0:1787784709.202466:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 166 00000100:00100000:0.0:1787784709.202477:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:23984:x1874627658041856:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1787784709.203142:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x15:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1787784709.203292:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967835 00010000:00010000:0.0:1787784709.203336:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.203352:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.203362:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.203369:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.203375:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.203380:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.203405:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787784709.203488:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23984:x1874627658041856:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1012us (1235us total) trans 4294967836 rc 0/0 00000100:00100000:0.0:1787784709.203503:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 166 00000020:00080000:0.0:1787784709.203509:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1787784709.203535:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 167 00000100:00100000:0.0:1787784709.203544:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507679850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+78:23983:x1874627658042112:12345-192.168.203.43@tcp:35:cat.0 00000001:00080000:0.0:1787784709.203683:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967837 00000100:00100000:0.0:1787784709.203733:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507679850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23983:x1874627658042112:12345-192.168.203.43@tcp:35:cat.0 Request processed in 191us (1425us total) trans 4294967838 rc 0/0 00000100:00100000:0.0:1787784709.203741:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 167 00000020:00080000:0.0:1787784709.203744:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1787784709.204131:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784709.204138:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.204180:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784709.204185:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.204195:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.204209:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff989503a6df00) 00010000:00010000:2.0:1787784709.204212:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787784709.204217:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.204224:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc5247 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.204240:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784709.204242:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784709.204244:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784709.227057:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784709.227646:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658042368 00000100:00100000:3.0:1787784709.227657:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658042368 00000100:00100000:2.0:1787784709.227996:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 822 00000100:00100000:2.0:1787784709.228007:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:23987:x1874627658042368:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784709.228033:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784709.228056:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc524e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b1a expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228188:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228203:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228215:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228230:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228237:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228243:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228592:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fca00/0xb0ce63137acc5263 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228601:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fca00/0xb0ce63137acc5263 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228609:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fca00/0xb0ce63137acc5263 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784709.228619:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:2.0:1787784709.228909:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228927:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228934:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228940:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784709.228945:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228952:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228966:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff989505542200) 00010000:00010000:2.0:1787784709.228969:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784709.228974:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228981:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5255 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228993:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.228999:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229005:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229009:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784709.229017:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229025:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229033:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4f0c1d45 (ffff989530cb1e00) 00010000:00010000:2.0:1787784709.229036:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784709.229039:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229045:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc525c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229084:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc524e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b1a expref: 83 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229098:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fca00/0xb0ce63137acc5263 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b1a expref: 82 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784709.229106:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:2.0:1787784709.229109:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784709.229359:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fca00, rc 0) 00000100:00100000:2.0:1787784709.229426:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:23987:x1874627658042368:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1420us (2377us total) trans 4294967839 rc 0/0 00000100:00100000:2.0:1787784709.229434:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 822 00000100:00100000:1.0:1787784709.232758:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.232879:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658042496 00000100:00100000:0.0:1787784709.232888:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658042496 00000100:00100000:0.0:1787784709.232947:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 823 00000100:00100000:0.0:1787784709.232956:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:23987:x1874627658042496:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:0.0:1787784709.233019:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233035:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233047:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233071:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233078:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233085:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233102:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784709.233107:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233115:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233140:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:0.0:1787784709.233143:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784709.233197:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.233204:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5271 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784709.233264:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:23987:x1874627658042496:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 309us (513us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784709.233272:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 823 00000100:00100000:3.0:1787784709.238442:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.238634:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658042752 00000100:00100000:0.0:1787784709.238644:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658042752 00000100:00100000:0.0:1787784709.238946:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 168 00000100:00100000:0.0:1787784709.238957:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:23987:x1874627658042752:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:0.0:1787784709.239326:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:23987:x1874627658042752:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 370us (890us total) trans 4294967840 rc 0/0 00000100:00100000:0.0:1787784709.239336:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 168 00000100:00100000:1.0:1787784709.374857:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784709.374969:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043008 00000100:00100000:0.0:1787784709.374979:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043008 00000100:00100000:0.0:1787784709.375038:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 824 00000100:00100000:0.0:1787784709.375062:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:24003:x1874627658043008:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:0.0:1787784709.375087:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784709.375106:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5278 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b21 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.375255:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc527f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.375271:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc527f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.375283:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc527f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784709.375295:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:0.0:1787784709.375419:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc5278 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b21 expref: 86 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.375437:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c17e00/0xb0ce63137acc527f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b21 expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784709.375445:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff989530cb1100) 00010000:00010000:0.0:1787784709.375447:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1787784709.375451:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c17e00, rc 0) 00000100:00100000:0.0:1787784709.375512:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:24003:x1874627658043008:12345-192.168.203.43@tcp:101:diff.0 Request processed in 453us (666us total) trans 4294967841 rc 0/0 00000100:00100000:0.0:1787784709.375519:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 824 00010000:00010000:0.0:1787784710.086940:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784710.642913:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784710.642946:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668374144:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784710.643040:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784710.643314:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668374144 00000100:00100000:3.0:1787784710.643324:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668374144 00000100:00100000:3.0:1787784710.643396:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:3.0:1787784710.643406:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668374144:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784710.643449:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 64684032 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784710.643501:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668374144:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (461us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784710.643508:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:3.0:1787784710.643561:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668374144:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787784710.643617:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784710.923241:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784710.923289:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784710.923315:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784710.923350:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784710.923365:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043520 00000100:00100000:1.0:1787784710.923375:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784710.923398:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043520 00000100:00100000:1.0:1787784710.923413:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784710.923450:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043776 00000100:00100000:1.0:1787784710.923455:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784710.923455:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043776 00000100:00100000:2.0:1787784710.923496:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043264 00000100:00100000:2.0:1787784710.923501:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043264 00000100:00100000:2.0:1787784710.923531:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043648 00000100:00100000:2.0:1787784710.923534:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043648 00000100:00100000:2.0:1787784710.923575:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00100000:2.0:1787784710.923584:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:2356:x1874627658043264:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784710.923665:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:2356:x1874627658043264:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 82us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784710.923673:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 321 00000100:00100000:1.0:1787784710.923684:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2356:x1874627658043648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784710.923718:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2356:x1874627658043648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784710.924635:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043392 00000100:00100000:0.0:1787784710.924646:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043392 00000100:00100000:0.0:1787784710.924686:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043904 00000100:00100000:0.0:1787784710.924688:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043904 00000100:00100000:0.0:1787784710.924703:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 269 00000100:00100000:0.0:1787784710.924711:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658043520:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784710.924753:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658043520:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (1463us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784710.924760:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 269 00000100:00100000:0.0:1787784710.924781:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 270 00000100:00100000:0.0:1787784710.924787:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658043776:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784710.924809:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658043776:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (1494us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784710.924815:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 270 00000100:00100000:0.0:1787784710.924826:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:0.0:1787784710.924892:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658043392:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784710.924917:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658043392:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (1542us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784710.924923:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:0.0:1787784710.924936:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 272 00000100:00100000:0.0:1787784710.924942:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658043904:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784710.924964:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658043904:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (1512us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784710.924983:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 272 00000100:00100000:0.0:1787784710.925010:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658043136 00000100:00100000:0.0:1787784710.925014:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658043136 00000100:00100000:0.0:1787784710.925124:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 531 00000100:00100000:0.0:1787784710.925131:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658043136:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784710.925185:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658043136:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 55us (1834us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784710.925191:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 531 00000100:00100000:2.0:1787784710.925357:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00080000:0.0:1787784710.925395:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784710.927347:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 321 00010000:00010000:0.0:1787784711.089601:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784712.118758:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784712.305986:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784712.305996:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784712.306300:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 20 count 2 00000040:00080000:1.0:1787784712.306307:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784712.689441:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668374272:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784712.689511:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784712.689575:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668374272 00000100:00100000:3.0:1787784712.689581:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668374272 00000100:00100000:3.0:1787784712.689625:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1787784712.689634:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668374272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784712.689695:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 64684032 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784712.689750:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668374272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784712.689757:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1787784712.689801:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668374272:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784712.689879:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668374400:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.689909:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784712.689931:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668374912:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.689956:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784712.689974:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784712.689980:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668374528:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.689999:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784712.690035:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668374400 00000100:00100000:3.0:1787784712.690038:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668374400 00000100:00100000:3.0:1787784712.690064:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 532 00000100:00100000:3.0:1787784712.690076:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668374400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690125:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668374400:12345-0@lo:400:kworker.0 Request processed in 54us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784712.690131:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 532 00000100:00100000:3.0:1787784712.690174:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668374912 00000100:00100000:3.0:1787784712.690178:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668374912 00000100:00100000:3.0:1787784712.690201:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668374528 00000100:00100000:3.0:1787784712.690204:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668374528 00000100:00100000:3.0:1787784712.690220:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1787784712.690226:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668374912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690255:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668374912:12345-0@lo:400:kworker.0 Request processed in 29us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784712.690268:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1787784712.690279:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 323 00000100:00100000:3.0:1787784712.690284:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668374528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690310:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668374528:12345-0@lo:400:kworker.0 Request processed in 26us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784712.690316:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 323 00000100:00100000:3.0:1787784712.690342:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668374528:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690350:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668374400:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690358:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668374912:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690377:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668374784:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690400:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784712.690417:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784712.690423:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668374656:0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690440:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784712.690458:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668374656 00000100:00100000:3.0:1787784712.690461:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668374656 00000100:00100000:3.0:1787784712.690484:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:3.0:1787784712.690489:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668374656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784712.690517:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668374656:12345-0@lo:400:kworker.0 Request processed in 27us (75us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784712.690522:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:3.0:1787784712.690542:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668374656:0@lo:400:kworker.0 00000100:00080000:1.0:1787784712.695470:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784712.695480:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784712.695524:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668374784 00000100:00100000:1.0:1787784712.695534:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668374784 00000100:00100000:1.0:1787784712.695591:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:1.0:1787784712.695601:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668374784:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1787784712.695761:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784712.698085:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668374784:0@lo:400:kworker.0 00000100:00100000:1.0:1787784712.698283:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668374784:12345-0@lo:400:kworker.0 Request processed in 2682us (7879us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784712.698295:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00010000:00010000:1.0:1787784713.137380:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784713.541829:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784713.541912:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658044032 00000100:00100000:1.0:1787784713.541922:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658044032 00000100:00100000:1.0:1787784713.542003:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 825 00000100:00100000:1.0:1787784713.542024:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23931:x1874627658044032:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1787784713.542128:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:23931:x1874627658044032:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 116us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784713.542176:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 825 00000100:00080000:3.0:1787784713.542242:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784714.162562:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784714.302744:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967841 is committed 00000001:00080000:1.0:1787784714.302761:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967836 is committed 00000020:00080000:1.0:1787784714.302772:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989518ed6180 refcount 2 committed 0 result 0 batchid 12 00000020:00080000:1.0:1787784714.302777:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989540610700 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989518ed69c0 00000020:00080000:1.0:1787784714.302781:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989540610900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff98953910d000 00000020:00080000:0.0:1787784714.302822:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 12 committed batchid 11 00000020:00080000:0.0:1787784714.302993:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 11 00000001:00080000:1.0:1787784714.313861:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967340 is committed 00000001:00080000:1.0:1787784714.313877:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967339 is committed 00000001:02000400:0.0:1787784715.029956:0:18424:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12c: Restore a file with stripe of 2 == 18:51:53 (1787784713) 00010000:00010000:3.0:1787784715.185457:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784715.434036:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.434288:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658044160 00000100:00100000:3.0:1787784715.434298:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658044160 00000100:00100000:3.0:1787784715.434384:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 826 00000100:00100000:3.0:1787784715.434402:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305161c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:24617:x1874627658044160:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1787784715.434509:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305161c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+35:24617:x1874627658044160:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 109us (481us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784715.434517:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 826 00000100:00080000:1.0:1787784715.434723:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784715.450472:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.450569:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658044288 00000100:00100000:2.0:1787784715.450578:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658044288 00000100:00100000:2.0:1787784715.450655:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 827 00000100:00100000:2.0:1787784715.450665:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:24618:x1874627658044288:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784715.450686:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784715.450707:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc528d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b28 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.450767:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.450781:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.450791:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.450805:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.450813:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.450819:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451094:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451103:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451110:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451115:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.451120:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451138:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451192:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:2.0:1787784715.451196:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.451201:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451207:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5294 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451219:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451225:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451230:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451241:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.451245:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451251:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451260:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff989530cb1700) 00010000:00010000:2.0:1787784715.451262:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.451265:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451270:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc529b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451299:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc528d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b28 expref: 81 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451310:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:2.0:1787784715.451312:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.451317:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc528d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b28 expref: 81 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.451326:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fdc00, rc 0) 00000100:00100000:2.0:1787784715.451383:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:24618:x1874627658044288:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 719us (922us total) trans 0 rc 301/301 00000100:00080000:1.0:1787784715.451493:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784715.453593:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 827 00000100:00100000:0.0:1787784715.461521:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.461607:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658044544 00000100:00100000:3.0:1787784715.461617:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658044544 00000100:00100000:3.0:1787784715.461721:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 828 00000100:00100000:3.0:1787784715.461732:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:24618:x1874627658044544:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:3.0:1787784715.461786:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784715.461815:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc508e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0a95 expref: 80 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.461833:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:3.0:1787784715.461839:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784715.462110:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.462125:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 34 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.462205:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.462250:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 34 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.462265:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 34 pid: 5811 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784715.462280:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668375040:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784715.462367:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063eec00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784715.466238:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.466344:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668375040:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784715.466381:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784715.466631:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658044672 00000100:00100000:1.0:1787784715.466638:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658044672 00010000:00010000:1.0:1787784715.466684:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 35 pid: 5811 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784715.466720:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 257 00000100:00100000:1.0:1787784715.466730:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:2356:x1874627658044672:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784715.466746:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784715.466752:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 35 pid: 5811 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.466764:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 35 pid: 5811 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.466785:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc50b8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0aa3 expref: 35 pid: 5811 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.466819:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:1.0:1787784715.466824:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.466833:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.466840:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.466872:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784715.466922:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2356:x1874627658044672:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 189us (689us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784715.466930:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 257 00010000:00010000:1.0:1787784715.467261:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.467268:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.467291:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.467300:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.467306:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784715.470958:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 512 00010000:00010000:1.0:1787784715.470985:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.470995:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.471002:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.471008:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784715.471012:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.471019:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.471035:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:1.0:1787784715.471039:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784715.471043:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.471048:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc52a2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784715.471068:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d9ee262e reply state 000000005bacf707 transno 4294967842 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784715.471092:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784715.473760:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:24618:x1874627658044544:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 12021us (12239us total) trans 4294967842 rc 0/0 00000100:00100000:1.0:1787784715.473773:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 828 00010000:00010000:1.0:1787784715.477483:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.477511:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.477519:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.477525:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784715.477529:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.477536:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.477553:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff98950a38cd00) 00010000:00010000:1.0:1787784715.477556:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784715.477560:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.477567:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc52a9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784715.498087:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.498508:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658044800 00000100:00100000:3.0:1787784715.498518:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658044800 00000100:00100000:3.0:1787784715.498628:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 829 00000100:00100000:3.0:1787784715.498640:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:24619:x1874627658044800:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784715.498784:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784715.498809:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc52b0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b2f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.498872:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc52b7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.498885:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc52b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.498909:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc52b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784715.498969:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784715.498974:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 23 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784715.498996:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc52b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.499020:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc52b0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b2f expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.499038:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc52b7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b2f expref: 81 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.499047:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:3.0:1787784715.499050:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784715.499052:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:3.0:1787784715.499123:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:24619:x1874627658044800:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 486us (1053us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784715.499365:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 829 00000100:00100000:0.0:1787784715.502100:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.502312:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658044928 00000100:00100000:2.0:1787784715.502336:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658044928 00000100:00100000:2.0:1787784715.502400:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 830 00000100:00100000:2.0:1787784715.502411:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:24619:x1874627658044928:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784715.502431:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784715.502451:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52be lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b36 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502519:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502533:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502544:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502559:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502567:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502573:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502698:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc52d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502707:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc52d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502714:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc52d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784715.502756:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 512 00000004:00200000:2.0:1787784715.502760:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784715.502782:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc52d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502795:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502801:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502808:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502814:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.502819:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502826:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502838:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784715.502841:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.502846:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502853:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52c5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502867:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502883:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502889:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502895:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.502898:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502904:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502912:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff989530cb1700) 00010000:00010000:2.0:1787784715.502915:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.502918:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502924:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52cc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502953:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52be lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b36 expref: 83 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502965:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc52d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b36 expref: 82 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.502973:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784715.502975:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784715.502984:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:2.0:1787784715.503045:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:24619:x1874627658044928:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 635us (968us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784715.503053:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 830 00000100:00100000:0.0:1787784715.507803:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.507880:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658045184 00000100:00100000:2.0:1787784715.507892:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658045184 00000100:00100000:2.0:1787784715.507948:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 831 00000100:00100000:2.0:1787784715.507958:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:24619:x1874627658045184:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784715.507977:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784715.507998:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc52da lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b3d expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508209:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508226:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508237:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508252:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508259:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508265:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508790:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 3/1,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508808:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508816:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784715.508826:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x37:0x0]: returning size 0 00010000:00010000:2.0:1787784715.508851:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508857:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508863:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508868:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fdc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.508872:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508880:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508893:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784715.508896:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.508900:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508906:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc52e1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784715.508924:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000017474d98 reply state 00000000548a7049 transno 4294967843 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784715.508939:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784715.508969:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc52da lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 85 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508981:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 84 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.508988:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1300) 00010000:00010000:2.0:1787784715.508990:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784715.508992:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc800, rc 0) 00000100:00100000:2.0:1787784715.509050:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:24619:x1874627658045184:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1093us (1255us total) trans 4294967843 rc 0/0 00000100:00100000:2.0:1787784715.509057:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 831 00010000:00010000:1.0:1787784715.511981:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.511998:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.512006:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.512012:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784715.512032:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.512040:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.512055:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff989530cb1700) 00010000:00010000:1.0:1787784715.512059:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784715.512063:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.512070:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc52e8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784715.515672:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.515762:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658045440 00000100:00100000:2.0:1787784715.515772:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658045440 00000100:00100000:2.0:1787784715.515829:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 832 00000100:00100000:2.0:1787784715.515839:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:24619:x1874627658045440:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784715.515864:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784715.515883:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc52fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b44 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.515976:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.515991:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 85 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.516019:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.516053:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 85 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.516067:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 85 pid: 5812 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784715.516091:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668375168:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784715.516570:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784715.525594:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784715.525676:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658045568 00000100:00100000:1.0:1787784715.525686:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658045568 00010000:00010000:1.0:1787784715.525723:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 86 pid: 5812 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784715.525772:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 258 00000100:00100000:1.0:1787784715.525781:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:2357:x1874627658045568:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784715.525798:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784715.525805:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 86 pid: 5812 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.525818:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 86 pid: 5812 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.525841:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1300) 00010000:00010000:1.0:1787784715.525859:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.525869:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.525875:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.525896:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784715.525971:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:2357:x1874627658045568:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 191us (387us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784715.525979:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 258 00010000:00010000:2.0:1787784715.527955:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc52ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b3d expref: 85 pid: 5812 timeout: 606 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784715.527982:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668375168:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784715.528005:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.528016:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.528021:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784715.528045:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x37:0x0]: returning size 0 00000004:00080000:2.0:1787784715.528418:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x12:0x0], index 1: 0 00000004:00080000:2.0:1787784715.528431:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x10:0x0], index 0: 0 00000100:00100000:0.0:1787784715.529255:0:8078:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504f00 pname:cluuid:pid:xid:nid:opc:job osp-pre-1-0:lustre-MDT0000-mdtlov_UUID:8078:1874627668375296:0@lo:5:osp-pre-1-0.0 00000100:00100000:0.0:1787784715.529320:0:8078:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784715.529359:0:8078:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503421480 going to sleep for 11 seconds 00000100:00100000:3.0:1787784715.529407:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668375296 00000100:00100000:3.0:1787784715.529414:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668375296 00000100:00100000:3.0:1787784715.529487:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:3.0:1787784715.529600:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:8078:x1874627668375296:12345-0@lo:5:osp-pre-1-0.0 00002000:00080000:3.0:1787784715.529640:0:10688:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0001: reserve 32 objects in group 0x280000400 at 34 00000100:00080000:0.0:1787784715.530090:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787784715.530381:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530391:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530398:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530403:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fdc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.530407:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530414:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530430:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1300) 00010000:00010000:2.0:1787784715.530433:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784715.530437:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530443:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5304 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530837:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc52fd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b44 expref: 85 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530851:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784715.530854:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784715.530858:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc52fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b44 expref: 85 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.530881:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc000, rc 0) 00000100:00100000:2.0:1787784715.530947:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:24619:x1874627658045440:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 15108us (15283us total) trans 4294967844 rc 301/301 00000100:00100000:2.0:1787784715.530956:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 832 00000100:00100000:0.0:1787784715.536608:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784715.541632:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658045824 00000100:00100000:1.0:1787784715.541645:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658045824 00000100:00100000:1.0:1787784715.541712:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 169 00000100:00100000:1.0:1787784715.541723:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:24619:x1874627658045824:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:0.0:1787784715.542828:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:24619:x1874627658045824:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 1101us (6221us total) trans 4294967845 rc 0/0 00000100:00100000:0.0:1787784715.542840:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 169 00000100:00100000:0.0:1787784715.546993:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.547069:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658045952 00000100:00100000:2.0:1787784715.547086:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658045952 00000100:00100000:2.0:1787784715.547181:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 833 00000100:00100000:2.0:1787784715.547192:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:24619:x1874627658045952:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784715.547215:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784715.547236:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc5312 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b4b expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.547293:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc5312 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b4b expref: 86 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.547308:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc5312 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b4b expref: 86 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.547328:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1900) 00010000:00010000:2.0:1787784715.547331:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784715.547334:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd400, rc 0) 00000100:00100000:2.0:1787784715.547384:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:24619:x1874627658045952:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 195us (405us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784715.547390:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 833 00000100:00100000:2.0:1787784715.557303:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.558538:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:8078:x1874627668375296:12345-0@lo:5:osp-pre-1-0.0 Request processed in 28935us (29212us total) trans 4294967341 rc 0/0 00000100:00100000:3.0:1787784715.558560:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1787784715.558724:0:8078:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967340 gen 2 00000004:00080000:2.0:1787784715.558730:0:8078:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967340 committed transno 4294967340 00000100:00100000:2.0:1787784715.558754:0:8078:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff989530504f00 x1874627668375296/t4294967341(4294967341) o5->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 432/432 e 0 to 0 dl 1787784731 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787784715.558766:0:8078:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504f00 pname:cluuid:pid:xid:nid:opc:job osp-pre-1-0:lustre-MDT0000-mdtlov_UUID:8078:1874627668375296:0@lo:5:osp-pre-1-0.0 00000004:00080000:2.0:1787784715.558780:0:8078:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0001-osc-MDT0000: current precreated pool: [0x280000400:0x12:0x0]-[0x280000400:0x41:0x0] 00000100:00100000:0.0:1787784715.560650:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658046208 00000100:00100000:0.0:1787784715.560665:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658046208 00000100:00100000:0.0:1787784715.560736:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 170 00000100:00100000:0.0:1787784715.560747:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:24619:x1874627658046208:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:0.0:1787784715.560966:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:24619:x1874627658046208:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 220us (3670us total) trans 4294967846 rc 0/0 00000100:00100000:0.0:1787784715.560975:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 170 00000100:00100000:0.0:1787784715.621661:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784715.621815:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658046336 00000100:00100000:1.0:1787784715.621825:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658046336 00000100:00100000:1.0:1787784715.621938:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 834 00000100:00100000:1.0:1787784715.621950:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:24625:x1874627658046336:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:1.0:1787784715.621978:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784715.621999:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc5319 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b52 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.622062:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc5320 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.622088:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc5320 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.622112:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc5320 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784715.622210:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 512 00000004:00200000:1.0:1787784715.622215:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784715.622239:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc5320 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.622280:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc5319 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b52 expref: 89 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.622298:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc5320 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b52 expref: 88 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.622306:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:1.0:1787784715.622309:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784715.622312:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641e00, rc 0) 00000100:00100000:1.0:1787784715.622371:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:24625:x1874627658046336:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 424us (718us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784715.622378:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 834 00000100:00100000:1.0:1787784715.636790:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.637048:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658046464 00000100:00100000:3.0:1787784715.637057:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658046464 00000100:00100000:3.0:1787784715.637130:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 835 00000100:00100000:3.0:1787784715.637140:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:24626:x1874627658046464:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784715.637209:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784715.637230:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc5327 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b59 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637295:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637309:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637321:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637338:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637345:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637351:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637521:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc533c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637547:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc533c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637555:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc533c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784715.637597:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x37:0x0]: returning size 0 00000004:00200000:3.0:1787784715.637602:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784715.637635:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc533c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637646:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc533c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637654:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637660:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637666:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637672:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784715.637676:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637683:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637694:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784715.637704:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784715.637709:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637715:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc532e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637729:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637736:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637742:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637746:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784715.637749:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637755:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637763:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff989505542d00) 00010000:00010000:3.0:1787784715.637765:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784715.637768:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.637774:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc5335 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.638292:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc5327 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b59 expref: 90 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.638320:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc533c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b59 expref: 89 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.638329:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1900) 00010000:00010000:3.0:1787784715.638331:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784715.638334:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351800, rc 0) 00000100:00100000:3.0:1787784715.638406:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:24626:x1874627658046464:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1267us (1623us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784715.638414:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 835 00000100:00100000:3.0:1787784715.643698:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784715.643835:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658046720 00000100:00100000:0.0:1787784715.643847:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658046720 00000100:00100000:0.0:1787784715.643924:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 206 00000100:00100000:0.0:1787784715.643935:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2354:x1874627658046720:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784715.643964:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784715.643984:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b67 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784715.644010:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x10:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784715.644029:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=506s v=(0 0) 00010000:00010000:0.0:1787784715.644034:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784715.644043:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784715.644092:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784715.644106:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16000, rc 0) 00000100:00080000:1.0:1787784715.644241:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784715.644354:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658046720:12345-192.168.203.43@tcp:101:rm.0 Request processed in 419us (664us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784715.644362:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 206 00000100:00100000:0.0:1787784715.647396:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784715.647509:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658046592 00000100:00100000:1.0:1787784715.647519:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658046592 00000100:00100000:1.0:1787784715.647590:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 207 00000100:00100000:1.0:1787784715.647601:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627658046592:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784715.647618:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784715.647652:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b60 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784715.647678:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x12:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784715.647696:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=506s v=(0 0) 00010000:00010000:1.0:1787784715.647701:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784715.647711:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.647728:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.647742:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641600, rc 0) 00000100:00100000:1.0:1787784715.647804:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658046592:12345-192.168.203.43@tcp:101:rm.0 Request processed in 205us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784715.647812:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 207 00000100:00080000:0.0:1787784715.648044:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784715.654259:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.654412:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658046976 00000100:00100000:2.0:1787784715.654422:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658046976 00000100:00100000:2.0:1787784715.654478:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 836 00000100:00100000:2.0:1787784715.654489:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:24626:x1874627658046976:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:2.0:1787784715.654541:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787784715.654569:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc533c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b59 expref: 89 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654593:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc5312 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b4b expref: 88 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654605:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1900) 00010000:00010000:2.0:1787784715.654622:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc5320 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b52 expref: 87 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654632:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784715.654635:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784715.654655:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654669:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654686:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654702:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654710:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654717:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654824:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654832:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.654840:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784715.655532:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].10, 64 off8904 00000004:00080000:2.0:1787784715.655542:0:10850:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 506742394652ns, next at 507742392427ns, rc = 0 00000040:00080000:2.0:1787784715.655583:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].7, 64 off8664 00000004:00080000:2.0:1787784715.655590:0:10850:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 506742442767ns, next at 507742440835ns, rc = 0 00010000:00010000:2.0:1787784715.655651:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655662:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655679:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655686:0:10850:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655693:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655699:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655724:0:10850:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787784715.655743:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655749:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655755:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655761:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.655766:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655773:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655782:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1d00) 00010000:00010000:2.0:1787784715.655785:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787784715.655796:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655802:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc535f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655818:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655825:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655830:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655836:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fdc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.655839:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655846:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655853:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784715.655855:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.655858:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.655863:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5351 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784715.655871:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008c10a163 reply state 0000000048c7812b transno 4294967847 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784715.655888:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784715.655960:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:24626:x1874627658046976:12345-192.168.203.43@tcp:36:rm.0 Request processed in 1472us (1708us total) trans 4294967847 rc 0/0 00000100:00100000:2.0:1787784715.655968:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 836 00010000:00010000:0.0:1787784715.656227:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784715.656234:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784715.656241:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784715.656244:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784715.656251:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784715.656260:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff989530cb1d00) 00010000:00010000:0.0:1787784715.656262:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787784715.656265:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784715.656270:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5366 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784715.656296:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784715.656297:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784715.656299:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784715.661194:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.661212:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.661220:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.661226:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784715.661231:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.661254:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.661271:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff989530cb1300) 00010000:00010000:1.0:1787784715.661274:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784715.661279:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.661286:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5358 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784715.674905:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.674975:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658047232 00000100:00100000:2.0:1787784715.674994:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658047232 00000100:00100000:2.0:1787784715.675058:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 837 00000100:00100000:2.0:1787784715.675068:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:24627:x1874627658047232:12345-192.168.203.43@tcp:101:dd.0 00010000:00010000:2.0:1787784715.675105:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784715.675127:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc536d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b6e expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.675471:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.675486:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.675508:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.675523:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.675531:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.675537:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784715.676604:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x13:0x0], index 1: 0 00010000:00010000:2.0:1787784715.677450:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677467:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677477:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677519:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677526:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677532:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677537:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784715.677541:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677549:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677560:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784715.677563:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784715.677567:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.677587:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5374 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784715.677600:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e28a3fa9 reply state 00000000b57fca3e transno 4294967848 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784715.677608:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784715.678521:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc536d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.678546:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 89 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.678555:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784715.678557:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784715.678560:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784715.678644:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:24627:x1874627658047232:12345-192.168.203.43@tcp:101:dd.0 Request processed in 3576us (3747us total) trans 4294967848 rc 0/0 00000100:00100000:2.0:1787784715.678653:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 837 00000100:00100000:2.0:1787784715.687567:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.688311:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658047360 00000100:00100000:3.0:1787784715.688323:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658047360 00000100:00100000:3.0:1787784715.688375:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 208 00000100:00100000:3.0:1787784715.688384:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:24627:x1874627658047360:12345-192.168.203.43@tcp:101:dd.0 00010000:00010000:3.0:1787784715.688401:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784715.688424:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b75 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784715.688449:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x13:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1787784715.688469:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=506s v=(0 0) 00010000:00010000:3.0:1787784715.688474:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784715.688484:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.688499:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.688527:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351400, rc 0) 00000100:00100000:3.0:1787784715.688583:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:24627:x1874627658047360:12345-192.168.203.43@tcp:101:dd.0 Request processed in 200us (1049us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784715.688591:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 208 00010000:00010000:1.0:1787784715.688885:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.688896:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.688904:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.688910:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784715.688915:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.688924:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.688937:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff989530cb1900) 00010000:00010000:1.0:1787784715.688941:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784715.688954:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784715.689036:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc537b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784715.724808:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.725008:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658047488 00000100:00100000:3.0:1787784715.725017:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658047488 00002000:00100000:3.0:1787784715.725067:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (0->1048575) req@ffff98951b5370c0 x1874627658047488/t0(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/0 e 0 to 0 dl 1787784726 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784715.725094:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5370c0 with x1874627658047488 ext(0->1048575) 00002000:00010000:3.0:1787784715.725100:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98951b5370c0 00000100:00100000:3.0:1787784715.725132:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 65 00000100:00100000:3.0:1787784715.725141:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5370c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658047488:12345-192.168.203.43@tcp:4:dd.0 00000100:00100000:3.0:1787784715.756351:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.762682:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784715.762705:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668375424:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784715.762840:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784715.762912:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668375424 00000100:00100000:3.0:1787784715.762920:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668375424 00000100:00100000:3.0:1787784715.762976:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1787784715.762985:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668375424:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784715.763019:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 64684032 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784715.763064:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668375424:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784715.763087:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1787784715.763245:0:6674:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 3 min 3 max 116 00000100:00100000:2.0:1787784715.763255:0:6674:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_003' 00000100:00100000:2.0:1787784715.763382:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668375424:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787784715.765641:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784715.765785:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658047616 00000100:00100000:1.0:1787784715.765796:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658047616 00002000:00100000:1.0:1787784715.765853:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (1048576->2097151) req@ffff98940400b840 x1874627658047616/t0(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/0 e 0 to 0 dl 1787784726 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.765865:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940400b840 with x1874627658047616 ext(1048576->2097151) 00002000:00010000:1.0:1787784715.765871:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98940400b840 00000100:00100000:1.0:1787784715.769887:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 66 00000100:00100000:1.0:1787784715.769908:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658047616:12345-192.168.203.43@tcp:4:dd.0 00000100:00100000:1.0:1787784715.774045:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5370c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658047488:12345-192.168.203.43@tcp:4:dd.0 Request processed in 48895us (49237us total) trans 4294967342 rc 0/0 00000100:00100000:1.0:1787784715.774061:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 65 00002000:00100000:1.0:1787784715.774083:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (0->1048575) req@ffff98951b5370c0 x1874627658047488/t4294967342(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/448 e 0 to 0 dl 1787784726 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.774094:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5370c0 with x1874627658047488 ext(0->1048575) 00002000:00010000:1.0:1787784715.774101:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98951b5370c0 00000100:00100000:3.0:1787784715.784837:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784715.785508:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658047744 00000100:00100000:1.0:1787784715.785518:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658047744 00002000:00100000:1.0:1787784715.785566:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (2097152->3145727) req@ffff9894040083c0 x1874627658047744/t0(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/0 e 0 to 0 dl 1787784726 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.785579:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894040083c0 with x1874627658047744 ext(2097152->3145727) 00002000:00010000:1.0:1787784715.785592:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894040083c0 00000100:00100000:1.0:1787784715.785627:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 67 00000100:00100000:1.0:1787784715.785638:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894040083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658047744:12345-192.168.203.43@tcp:4:dd.0 00000100:00100000:1.0:1787784715.792423:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+19:2357:x1874627658047616:12345-192.168.203.43@tcp:4:dd.0 Request processed in 22513us (36071us total) trans 4294967343 rc 0/0 00000100:00100000:1.0:1787784715.792948:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 66 00002000:00100000:1.0:1787784715.792962:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x13:0x0] (1048576->2097151) req@ffff98940400b840 x1874627658047616/t4294967343(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/448 e 0 to 0 dl 1787784726 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.792972:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940400b840 with x1874627658047616 ext(1048576->2097151) 00002000:00010000:1.0:1787784715.792978:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98940400b840 00000100:00100000:2.0:1787784715.794961:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784715.796339:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658047872 00000100:00100000:1.0:1787784715.796352:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658047872 00002000:00100000:1.0:1787784715.796409:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (3145728->4194303) req@ffff9895305170c0 x1874627658047872/t0(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/0 e 0 to 0 dl 1787784726 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.796422:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895305170c0 with x1874627658047872 ext(3145728->4194303) 00002000:00010000:1.0:1787784715.796588:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9895305170c0 00000100:00100000:1.0:1787784715.796623:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 68 00000100:00100000:1.0:1787784715.796634:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305170c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+19:2356:x1874627658047872:12345-192.168.203.43@tcp:4:dd.0 00000100:00100000:1.0:1787784715.821618:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894040083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+21:2354:x1874627658047744:12345-192.168.203.43@tcp:4:dd.0 Request processed in 35968us (36777us total) trans 4294967344 rc 0/0 00000100:00100000:1.0:1787784715.821635:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 67 00002000:00100000:1.0:1787784715.821648:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (2097152->3145727) req@ffff9894040083c0 x1874627658047744/t4294967344(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/448 e 0 to 0 dl 1787784726 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.821681:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894040083c0 with x1874627658047744 ext(2097152->3145727) 00002000:00010000:1.0:1787784715.821687:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894040083c0 00000100:00100000:2.0:1787784715.825653:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.825695:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784715.825814:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658048000 00000100:00100000:1.0:1787784715.825824:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658048000 00002000:00100000:1.0:1787784715.825872:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x13:0x0] (4194304->5242879) req@ffff98953f7b0000 x1874627658048000/t0(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/0 e 0 to 0 dl 1787784726 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.825885:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f7b0000 with x1874627658048000 ext(4194304->5242879) 00002000:00010000:1.0:1787784715.825891:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98953f7b0000 00000100:00100000:1.0:1787784715.825927:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 69 00000100:00100000:1.0:1787784715.825937:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+21:2356:x1874627658048000:12345-192.168.203.43@tcp:4:dd.0 00000100:00100000:0.0:1787784715.827936:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658048256 00000100:00100000:0.0:1787784715.827948:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658048256 00000100:00100000:0.0:1787784715.828008:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 171 00000100:00100000:0.0:1787784715.828018:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:24627:x1874627658048256:12345-192.168.203.43@tcp:35:dd.0 00000100:00100000:0.0:1787784715.830396:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:24627:x1874627658048256:12345-192.168.203.43@tcp:35:dd.0 Request processed in 2370us (4691us total) trans 4294967849 rc 0/0 00000100:00100000:0.0:1787784715.830408:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 171 00000100:00100000:3.0:1787784715.843445:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305170c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+23:2356:x1874627658047872:12345-192.168.203.43@tcp:4:dd.0 Request processed in 46794us (48472us total) trans 4294967345 rc 0/0 00000100:00100000:3.0:1787784715.843460:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 68 00002000:00100000:3.0:1787784715.843476:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (3145728->4194303) req@ffff9895305170c0 x1874627658047872/t4294967345(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/448 e 0 to 0 dl 1787784726 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784715.843486:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895305170c0 with x1874627658047872 ext(3145728->4194303) 00002000:00010000:3.0:1787784715.843494:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9895305170c0 00000100:00100000:1.0:1787784715.861033:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+23:2356:x1874627658048000:12345-192.168.203.43@tcp:4:dd.0 Request processed in 35088us (35380us total) trans 4294967346 rc 0/0 00000100:00100000:1.0:1787784715.861046:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 69 00002000:00100000:1.0:1787784715.861058:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x13:0x0] (4194304->5242879) req@ffff98953f7b0000 x1874627658048000/t4294967346(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:596/0 lens 488/448 e 0 to 0 dl 1787784726 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784715.861068:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f7b0000 with x1874627658048000 ext(4194304->5242879) 00002000:00010000:1.0:1787784715.861084:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98953f7b0000 00000100:00100000:0.0:1787784715.911583:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.911677:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658048512 00000100:00100000:2.0:1787784715.911700:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658048512 00000100:00100000:2.0:1787784715.911762:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 838 00000100:00100000:2.0:1787784715.911772:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:24630:x1874627658048512:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:2.0:1787784715.911796:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784715.911817:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc5397 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b7c expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.911913:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc539e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.911927:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc539e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.911938:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc539e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784715.911948:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:2.0:1787784715.912071:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc5397 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b7c expref: 93 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.912103:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc539e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b7c expref: 92 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784715.912110:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784715.912113:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1787784715.912116:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784715.912234:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:24630:x1874627658048512:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 464us (660us total) trans 4294967850 rc 0/0 00000100:00100000:2.0:1787784715.912242:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 838 00000100:00100000:2.0:1787784715.977621:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784715.977743:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658048640 00000100:00100000:3.0:1787784715.977754:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658048640 00000100:00100000:3.0:1787784715.977810:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 839 00000100:00100000:3.0:1787784715.977819:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:24631:x1874627658048640:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784715.977843:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784715.977863:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc53ac lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b83 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.977950:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.977964:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.977977:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784715.978033:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 512 00000004:00200000:3.0:1787784715.978036:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784715.978071:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.978112:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc53ac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 94 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.978130:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 93 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784715.978138:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784715.978141:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784715.978144:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403350c00, rc 0) 00000100:00100000:3.0:1787784715.978273:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:24631:x1874627658048640:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 454us (661us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784715.978280:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 839 00000100:00100000:3.0:1787784715.983434:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784715.983604:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658048896 00000100:00100000:2.0:1787784715.983614:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658048896 00000100:00100000:2.0:1787784715.983676:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 840 00000100:00100000:2.0:1787784715.983687:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:24631:x1874627658048896:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:2.0:1787784715.983948:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].20, 136 off10912 00000100:00100000:2.0:1787784715.984046:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:24631:x1874627658048896:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 359us (621us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784715.984054:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 840 00000100:00100000:1.0:1787784716.046418:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.046463:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.046496:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.046529:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.046557:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658049280 00000100:00100000:1.0:1787784716.046566:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658049280 00000100:00100000:1.0:1787784716.046620:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658049408 00000100:00100000:1.0:1787784716.046637:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658049408 00000100:00100000:1.0:1787784716.046666:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 274 00000100:00100000:1.0:1787784716.046675:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658049280:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784716.046751:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658049280:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 76us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784716.046768:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 275 00000100:00100000:3.0:1787784716.046792:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658049408:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784716.046814:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658049024 00000100:00100000:2.0:1787784716.046821:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658049024 00000100:00100000:2.0:1787784716.046861:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 533 00000100:00100000:2.0:1787784716.046872:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658049024:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784716.046882:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658049408:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 94us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.046996:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658049024:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 125us (584us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.047005:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 533 00000100:00080000:2.0:1787784716.047038:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784716.047042:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784716.047069:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658049152 00000100:00100000:2.0:1787784716.047086:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658049152 00000100:00100000:2.0:1787784716.047125:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:2.0:1787784716.047133:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2354:x1874627658049152:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784716.047200:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2354:x1874627658049152:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 68us (703us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.047217:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:1.0:1787784716.048592:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 274 00000100:00100000:3.0:1787784716.056973:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 275 00010000:00010000:0.0:1787784716.209791:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784716.401398:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784716.401408:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784716.401447:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 21 count 3 00000040:00080000:0.0:1787784716.401454:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784716.401707:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a91e00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668375552:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784716.401794:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503421c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784716.410265:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.410392:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658049536 00000100:00100000:1.0:1787784716.410401:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658049536 00000100:00100000:1.0:1787784716.410459:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 841 00000100:00100000:1.0:1787784716.410469:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:24641:x1874627658049536:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784716.410500:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784716.410613:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:24641:x1874627658049536:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 145us (356us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784716.410622:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 841 00000100:00100000:0.0:1787784716.410899:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a91e00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668375552:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784716.416594:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.416681:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658049792 00000100:00100000:1.0:1787784716.416690:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658049792 00000100:00100000:1.0:1787784716.416758:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 842 00000100:00100000:1.0:1787784716.416767:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+34:24641:x1874627658049792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784716.416786:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784716.416807:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc53ba lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b8a expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.416911:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.416925:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.416954:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784716.416967:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:1.0:1787784716.417108:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc53ba lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.417127:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.417137:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:1.0:1787784716.417139:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784716.417143:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784716.417272:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:24641:x1874627658049792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 505us (685us total) trans 4294967851 rc 0/0 00000100:00100000:1.0:1787784716.417279:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 842 00000100:00100000:1.0:1787784716.433134:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.433183:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658049920 00000100:00100000:2.0:1787784716.433205:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658049920 00000100:00100000:2.0:1787784716.433261:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 209 00000100:00100000:2.0:1787784716.433276:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658049920:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784716.433323:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/1,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.433346:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 22 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.433356:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x13:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784716.433360:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784716.433364:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 4/1,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.433402:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 22 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.433416:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 22 pid: 10688 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.433431:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031243c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668375680:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784716.433518:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef480 going to sleep for 11 seconds 00000100:00100000:1.0:1787784716.440671:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.440704:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658050048 00000100:00100000:1.0:1787784716.440713:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658050048 00010000:00010000:1.0:1787784716.440763:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 23 pid: 10688 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784716.440801:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 259 00000100:00100000:1.0:1787784716.440810:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ec050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+23:2356:x1874627658050048:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784716.440823:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1787784716.440859:0:5796:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb size from disk: 0 -> 5242880 00002000:00010000:1.0:1787784716.440863:0:5796:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb mtime from disk: 0 -> 1787784715 00002000:00010000:1.0:1787784716.440866:0:5796:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb ctime from disk: 0 -> 1787784715 00010000:00010000:1.0:1787784716.440873:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 23 pid: 10688 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.440889:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 23 pid: 10688 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.440915:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784716.440920:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784716.440926:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.440937:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.440961:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784716.441010:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ec050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+23:2356:x1874627658050048:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 200us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784716.441019:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 259 00010000:00010000:2.0:1787784716.441944:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc5390 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b75 expref: 22 pid: 10688 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.441961:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031243c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668375680:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784716.441988:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442000:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442007:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442034:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442041:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442049:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442055:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fdc00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784716.442059:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442066:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442091:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.442105:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c303bb98/0xb0ce63137acc53cf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.442202:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658049920:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 8932us (9078us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.442211:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 209 00000100:00100000:1.0:1787784716.446787:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.446998:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658050304 00000100:00100000:2.0:1787784716.447007:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658050304 00000100:00100000:2.0:1787784716.447111:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 843 00000100:00100000:2.0:1787784716.447121:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+36:24641:x1874627658050304:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784716.447257:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+36:24641:x1874627658050304:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 136us (479us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.447265:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 843 00000100:00100000:3.0:1787784716.452659:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.452747:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658050432 00000100:00100000:0.0:1787784716.452756:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658050432 00000100:00100000:0.0:1787784716.452805:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:0.0:1787784716.452815:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658050432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784716.452832:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784716.452860:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b91 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784716.452886:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x13:0x0] initial LVB size: 5242880, mtime: 0x6a8f6e0b, atime: 0x0, ctime: 0x6a8f6e0b, blocks: 0x1 00010000:00010000:0.0:1787784716.452905:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=507s v=(0 0) 00010000:00010000:0.0:1787784716.452909:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784716.452919:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.452936:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.452949:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:0.0:1787784716.453011:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658050432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 197us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784716.453018:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:0.0:1787784716.457338:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.457432:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658050688 00000100:00100000:1.0:1787784716.457441:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658050688 00000100:00100000:1.0:1787784716.457495:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 844 00000100:00100000:1.0:1787784716.457505:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:24641:x1874627658050688:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784716.457653:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:24641:x1874627658050688:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 148us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784716.457730:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 844 00000100:00100000:1.0:1787784716.466038:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.466194:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658050816 00000100:00100000:0.0:1787784716.466205:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658050816 00002000:00100000:0.0:1787784716.466256:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (0->1048575) req@ffff989530515680 x1874627658050816/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/0 e 0 to 0 dl 1787784727 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784716.466266:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530515680 with x1874627658050816 ext(0->1048575) 00002000:00010000:0.0:1787784716.466272:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530515680 00000100:00100000:0.0:1787784716.466307:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 70 00000100:00100000:0.0:1787784716.466316:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658050816:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784716.502609:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658050816:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 36285us (36571us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1787784716.502623:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 70 00002000:00100000:2.0:1787784716.502636:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (0->1048575) req@ffff989530515680 x1874627658050816/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/440 e 0 to 0 dl 1787784727 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784716.502646:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530515680 with x1874627658050816 ext(0->1048575) 00002000:00010000:2.0:1787784716.502653:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530515680 00000100:00100000:3.0:1787784716.535693:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.535780:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658050944 00000100:00100000:0.0:1787784716.535796:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658050944 00002000:00100000:0.0:1787784716.535840:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (1048576->2097151) req@ffff989539378f00 x1874627658050944/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/0 e 0 to 0 dl 1787784727 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784716.535852:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539378f00 with x1874627658050944 ext(1048576->2097151) 00002000:00010000:0.0:1787784716.535857:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539378f00 00000100:00100000:0.0:1787784716.535886:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 71 00000100:00100000:0.0:1787784716.535895:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539378f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658050944:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1787784716.565988:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539378f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658050944:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 30083us (30293us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1787784716.566005:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 71 00002000:00100000:0.0:1787784716.566018:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (1048576->2097151) req@ffff989539378f00 x1874627658050944/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/440 e 0 to 0 dl 1787784727 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784716.566029:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539378f00 with x1874627658050944 ext(1048576->2097151) 00002000:00010000:0.0:1787784716.566035:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539378f00 00000100:00100000:3.0:1787784716.581620:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.581739:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658051072 00000100:00100000:2.0:1787784716.581758:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658051072 00002000:00100000:2.0:1787784716.581807:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x13:0x0] (2097152->3145727) req@ffff989539379680 x1874627658051072/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/0 e 0 to 0 dl 1787784727 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784716.581840:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539379680 with x1874627658051072 ext(2097152->3145727) 00002000:00010000:2.0:1787784716.581845:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539379680 00000100:00100000:2.0:1787784716.581890:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 72 00000100:00100000:2.0:1787784716.581900:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658051072:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1787784716.595422:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.597032:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658051200 00000100:00100000:2.0:1787784716.597047:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658051200 00002000:00100000:2.0:1787784716.597110:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x13:0x0] (3145728->4194303) req@ffff989530515e00 x1874627658051200/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/0 e 0 to 0 dl 1787784727 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784716.597121:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530515e00 with x1874627658051200 ext(3145728->4194303) 00002000:00010000:2.0:1787784716.597126:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530515e00 00000100:00100000:2.0:1787784716.597241:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 73 00000100:00100000:2.0:1787784716.597250:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658051200:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1787784716.611234:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539379680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658051072:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 29325us (29612us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1787784716.611249:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 72 00002000:00100000:1.0:1787784716.611262:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x13:0x0] (2097152->3145727) req@ffff989539379680 x1874627658051072/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/440 e 0 to 0 dl 1787784727 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784716.611275:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539379680 with x1874627658051072 ext(2097152->3145727) 00002000:00010000:1.0:1787784716.611281:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539379680 00000100:00100000:2.0:1787784716.619518:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658051200:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 22258us (24094us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1787784716.619532:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 73 00002000:00100000:2.0:1787784716.619545:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x13:0x0] (3145728->4194303) req@ffff989530515e00 x1874627658051200/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/440 e 0 to 0 dl 1787784727 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784716.619555:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530515e00 with x1874627658051200 ext(3145728->4194303) 00002000:00010000:2.0:1787784716.619562:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530515e00 00000100:00100000:3.0:1787784716.651732:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.651809:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658051328 00000100:00100000:2.0:1787784716.651819:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658051328 00002000:00100000:2.0:1787784716.651864:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (4194304->5242879) req@ffff9894011ee1c0 x1874627658051328/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/0 e 0 to 0 dl 1787784727 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784716.651887:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894011ee1c0 with x1874627658051328 ext(4194304->5242879) 00002000:00010000:2.0:1787784716.651892:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894011ee1c0 00000100:00100000:2.0:1787784716.651956:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 74 00000100:00100000:2.0:1787784716.651966:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658051328:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784716.679803:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658051328:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 27829us (28071us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1787784716.679817:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 74 00002000:00100000:2.0:1787784716.679830:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (4194304->5242879) req@ffff9894011ee1c0 x1874627658051328/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:597/0 lens 488/440 e 0 to 0 dl 1787784727 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784716.679840:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894011ee1c0 with x1874627658051328 ext(4194304->5242879) 00002000:00010000:2.0:1787784716.679846:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894011ee1c0 00000100:00100000:1.0:1787784716.701188:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.701225:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658051456 00000100:00100000:1.0:1787784716.701234:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658051456 00000100:00100000:1.0:1787784716.701279:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 845 00000100:00100000:1.0:1787784716.701288:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+36:24641:x1874627658051456:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784716.701306:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784716.701324:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc53dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b98 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.701358:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.701372:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.701524:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.701628:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc53dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 38 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.701643:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 37 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.701651:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:1.0:1787784716.701654:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784716.701658:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784716.702830:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24641:x1874627658051456:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1535us (1643us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784716.702841:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 845 00000100:00100000:0.0:1787784716.711603:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.711694:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658051584 00000100:00100000:2.0:1787784716.711703:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658051584 00000100:00100000:2.0:1787784716.711765:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 846 00000100:00100000:2.0:1787784716.711776:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24641:x1874627658051584:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784716.711806:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784716.711915:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24641:x1874627658051584:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 141us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.711924:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 846 00000100:00100000:2.0:1787784716.717166:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784716.717241:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658051712 00000100:00100000:3.0:1787784716.717250:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658051712 00000100:00100000:3.0:1787784716.717303:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 211 00000100:00100000:3.0:1787784716.717313:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658051712:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784716.717374:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 3/1,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717401:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784716.717405:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784716.717411:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717426:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717456:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717463:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717472:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717478:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784716.717482:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717513:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717527:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.717535:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc53eb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784716.717614:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658051712:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 302us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784716.717623:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 211 00000100:00100000:1.0:1787784716.720234:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.720417:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658051968 00000100:00100000:2.0:1787784716.720426:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658051968 00000100:00100000:2.0:1787784716.720481:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 847 00000100:00100000:2.0:1787784716.720503:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24641:x1874627658051968:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784716.720587:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.720605:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 37 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.720616:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.720648:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 37 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.720676:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 37 pid: 5813 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.720690:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668375808:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784716.720773:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784716.731918:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668375808:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784716.731947:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784716.732106:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.732217:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658052096 00000100:00100000:2.0:1787784716.732224:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658052096 00010000:00010000:2.0:1787784716.732259:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 38 pid: 5813 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.732293:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 260 00000100:00100000:2.0:1787784716.732303:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2357:x1874627658052096:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784716.732318:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784716.732324:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 38 pid: 5813 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.732344:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 38 pid: 5813 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.732366:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc53e4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b98 expref: 38 pid: 5813 timeout: 607 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.732390:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784716.732396:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.732406:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.732414:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.732441:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784716.732473:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.732481:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.732495:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2357:x1874627658052096:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 194us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.732503:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 260 00010000:00010000:3.0:1787784716.732682:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.732691:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.732704:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.732711:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784716.732715:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.732724:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.732732:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:3.0:1787784716.732735:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784716.732738:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.732745:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc53f2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784716.732877:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24641:x1874627658051968:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 12376us (12649us total) trans 4294967852 rc 0/0 00000100:00100000:3.0:1787784716.732886:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 847 00000040:00080000:2.0:1787784716.733265:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784716.733270:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:3.0:1787784716.736230:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.736435:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658052352 00000100:00100000:0.0:1787784716.736446:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658052352 00000100:00100000:0.0:1787784716.736500:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 172 00000100:00100000:0.0:1787784716.736508:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24641:x1874627658052352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000040:00080000:2.0:1787784716.736626:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1787784716.736638:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784716.736734:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24641:x1874627658052352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 225us (512us total) trans 4294967853 rc 0/0 00000100:00100000:0.0:1787784716.736742:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 172 00000040:00080000:0.0:1787784716.774267:0:18475:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784716.774283:0:18475:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784716.774332:0:18475:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 21 count 3 00000040:00080000:0.0:1787784716.774339:0:18475:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784716.870136:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.870194:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658052480 00000100:00100000:0.0:1787784716.870202:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658052480 00000100:00100000:0.0:1787784716.870251:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 212 00000100:00100000:0.0:1787784716.870261:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+22:2356:x1874627658052480:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784716.870280:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784716.870299:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0b9f expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.870325:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784716.870330:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784716.870340:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 23 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.870357:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 23 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.870370:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e2b000, rc 0) 00000100:00100000:0.0:1787784716.870430:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+23:2356:x1874627658052480:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 169us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784716.870438:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 212 00000100:00100000:0.0:1787784716.883056:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.885950:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658052736 00000100:00100000:2.0:1787784716.885966:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658052736 00000100:00100000:2.0:1787784716.886026:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:2.0:1787784716.886036:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d6050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:24642:x1874627658052736:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:2.0:1787784716.886617:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d6050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:24642:x1874627658052736:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 580us (3565us total) trans 4294967854 rc 0/0 00000100:00100000:2.0:1787784716.886626:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 173 00000100:00100000:3.0:1787784716.891907:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.892632:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658052992 00000100:00100000:2.0:1787784716.892643:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658052992 00000100:00100000:2.0:1787784716.892842:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 848 00000100:00100000:2.0:1787784716.892853:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:24642:x1874627658052992:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784716.892878:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784716.892881:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787784716.892908:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc539e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b7c expref: 94 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.893000:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784716.893006:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784716.893021:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5400 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ba6 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.893524:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5407 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.893539:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5407 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.893550:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5407 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784716.893561:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:2.0:1787784716.894118:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5400 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ba6 expref: 96 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.894238:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5407 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ba6 expref: 95 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.894248:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784716.894252:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1787784716.894254:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784716.894329:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:24642:x1874627658052992:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1477us (2430us total) trans 4294967855 rc 0/0 00000100:00100000:2.0:1787784716.894338:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 848 00000100:00100000:1.0:1787784716.899385:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.899499:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658053120 00000100:00100000:2.0:1787784716.899508:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658053120 00000100:00100000:2.0:1787784716.899570:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 213 00000100:00100000:2.0:1787784716.899581:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+23:2357:x1874627658053120:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:2.0:1787784716.899632:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.899652:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.899662:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 23 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.899669:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784716.899673:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784716.899678:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.899722:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 23 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.899736:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 23 pid: 6670 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.899751:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668375936:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784716.899859:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.899869:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 9 pid: 10688 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.899879:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668376064:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784716.899902:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784716.910241:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668375936:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784716.915783:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.915950:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668376064:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784716.915979:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784716.919048:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658053248 00000100:00100000:1.0:1787784716.919062:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658053248 00010000:00010000:1.0:1787784716.919109:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 24 pid: 6670 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784716.919169:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 261 00000100:00100000:1.0:1787784716.919180:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+24:2357:x1874627658053248:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784716.919196:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784716.919218:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 24 pid: 6670 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.919247:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 24 pid: 6670 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.919272:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094e65c35/0xb0ce63137acc53f9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b9f expref: 24 pid: 6670 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.919288:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784716.919509:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+23:2357:x1874627658053248:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 329us (3731us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784716.919518:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 261 00000100:00100000:3.0:1787784716.923006:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.923168:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658053376 00000100:00100000:0.0:1787784716.923178:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658053376 00010000:00010000:0.0:1787784716.923228:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 10 pid: 10688 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784716.924221:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 262 00000100:00100000:0.0:1787784716.924235:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658053376:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784716.924253:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784716.924278:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 10 pid: 10688 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.924296:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 10 pid: 10688 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.924319:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006390a6a9/0xb0ce63137acc53d6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b91 expref: 10 pid: 10688 timeout: 607 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.924349:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784716.924354:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784716.924361:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.924371:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.924510:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784716.924562:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658053376:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 330us (1564us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784716.924581:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 262 00010000:00010000:3.0:1787784716.924647:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924658:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924690:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924699:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924708:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924716:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fde00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784716.924720:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924728:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924740:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784716.924748:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c9b5213a/0xb0ce63137acc5415 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784716.924796:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+22:2357:x1874627658053120:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 25217us (25421us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784716.924804:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 213 00000100:00100000:3.0:1787784716.930742:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.930817:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658053632 00000100:00100000:0.0:1787784716.930826:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658053632 00000100:00100000:0.0:1787784716.930881:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 174 00000100:00100000:0.0:1787784716.930892:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:24642:x1874627658053632:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:0.0:1787784716.930964:0:5815:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000401:0x38:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5407 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ba6 expref: 95 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784716.931228:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967856 00000001:00080000:0.0:1787784716.932122:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967857 00010000:00010000:0.0:1787784716.932227:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.932246:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 98 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.932257:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.932266:0:5815:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 5/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.932300:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.932316:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 37 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784716.932355:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668376192:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784716.934104:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 98 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.934123:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 98 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784716.934139:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a91a40 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668376320:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784716.934204:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895034216c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784716.940434:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.940476:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784716.940554:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668376192:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784716.940569:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a91a40 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668376320:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784716.940603:0:5815:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.941350:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658053888 00000100:00100000:2.0:1787784716.941359:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658053888 00010000:00010000:2.0:1787784716.941405:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 99 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.941438:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658053760 00000100:00100000:2.0:1787784716.941443:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658053760 00010000:00010000:2.0:1787784716.941455:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 38 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.941546:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:2.0:1787784716.941558:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:2357:x1874627658053888:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784716.941573:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787784716.941578:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 264 00010000:00010000:2.0:1787784716.941584:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 99 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784716.941592:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2357:x1874627658053760:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784716.941601:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 99 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.941603:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784716.941614:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 38 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.941697:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5382 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b6e expref: 99 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.941827:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 38 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.941837:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784716.941845:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.941853:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.941859:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784716.941869:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc53c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b8a expref: 38 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.941882:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:1.0:1787784716.941894:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.941903:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.941910:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784716.941964:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:2357:x1874627658053888:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 407us (1538us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.941983:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 263 00010000:00010000:1.0:1787784716.942007:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784716.942032:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2357:x1874627658053760:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 444us (1555us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784716.942040:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 264 00010000:00010000:1.0:1787784716.942132:0:5815:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.942140:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784716.942279:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967858 00000001:00080000:1.0:1787784716.944921:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967859 00000100:00100000:1.0:1787784716.947015:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09f840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668376448:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1787784716.947176:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784716.947209:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063eb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784716.947277:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668376448 00000100:00100000:0.0:1787784716.947285:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668376448 00000100:00100000:0.0:1787784716.947491:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 29 00000100:00100000:0.0:1787784716.947501:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668376448:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1787784716.947574:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668376448:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 74us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784716.947581:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 29 00000100:00100000:1.0:1787784716.947646:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668376448:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:1.0:1787784716.948038:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967860 00000100:00100000:1.0:1787784716.948315:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09f840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668376576:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1787784716.949187:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784716.949219:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d300 going to sleep for 11 seconds 00000100:00100000:2.0:1787784716.949342:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668376576 00000100:00100000:2.0:1787784716.949350:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668376576 00000100:00100000:2.0:1787784716.949395:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30 00000100:00100000:2.0:1787784716.949404:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668376576:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1787784716.949595:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5815:x1874627668376576:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 192us (409us total) trans 4294967347 rc 0/0 00000100:00100000:2.0:1787784716.949603:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30 00000100:00100000:1.0:1787784716.949701:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668376576:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:1.0:1787784716.949780:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967861 00010000:00010000:1.0:1787784716.949815:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.949825:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.949832:0:5815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.949838:0:5815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e2a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784716.949842:0:5815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.949850:0:5815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.949866:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:1.0:1787784716.949869:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:1.0:1787784716.949874:0:5815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.949880:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a200/0xb0ce63137acc541c lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787784716.950060:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x39:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1787784716.950233:0:5815:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].11, 64 off8968 00000004:00080000:1.0:1787784716.950244:0:5815:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 508037096318ns, next at 509037093961ns, rc = 0 00000001:00080000:1.0:1787784716.950250:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967862 00010000:00010000:1.0:1787784716.950280:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989530cb1500) 00010000:00010000:1.0:1787784716.950283:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1787784716.950296:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5407 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ba6 expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784716.950692:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:24642:x1874627658053632:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 19799us (19956us total) trans 4294967863 rc 0/0 00000100:00100000:1.0:1787784716.950702:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 174 00000020:00080000:1.0:1787784716.950706:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:0.0:1787784716.971448:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784716.971489:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658054144 00000100:00100000:0.0:1787784716.971499:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658054144 00000100:00100000:0.0:1787784716.971780:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 849 00000100:00100000:0.0:1787784716.973121:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:24645:x1874627658054144:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:0.0:1787784716.973248:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784716.973272:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ae00/0xb0ce63137acc5423 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bad expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973363:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973377:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973388:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973403:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973411:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973418:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973559:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a600/0xb0ce63137acc5438 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973568:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a600/0xb0ce63137acc5438 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.973575:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a600/0xb0ce63137acc5438 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784716.973585:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:0.0:1787784716.974059:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974089:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974095:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974102:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e2aa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784716.974105:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974112:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974124:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:0.0:1787784716.974127:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784716.974221:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974228:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2aa00/0xb0ce63137acc542a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974241:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974247:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974253:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974257:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e2ac00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784716.974266:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974273:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974282:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff989503a6de00) 00010000:00010000:0.0:1787784716.974285:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784716.974288:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974293:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ac00/0xb0ce63137acc5431 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974338:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2ae00/0xb0ce63137acc5423 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bad expref: 104 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974353:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a600/0xb0ce63137acc5438 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bad expref: 103 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784716.974361:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:0.0:1787784716.974363:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784716.974366:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e2a600, rc 0) 00000100:00100000:0.0:1787784716.974428:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:24645:x1874627658054144:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 2638us (2988us total) trans 4294967864 rc 0/0 00000100:00100000:0.0:1787784716.974437:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 849 00000100:00100000:3.0:1787784716.990958:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784716.991035:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658054400 00000100:00100000:2.0:1787784716.991044:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658054400 00000100:00100000:2.0:1787784716.991122:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 34 00000100:00100000:2.0:1787784716.991133:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:24645:x1874627658054400:12345-192.168.203.43@tcp:103:md5sum.0 00010000:00010000:2.0:1787784716.991173:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784716.991197:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2a600/0xb0ce63137acc5438 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bad expref: 103 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784716.991213:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989503a6dc00) 00010000:00010000:2.0:1787784716.991224:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784716.991276:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:24645:x1874627658054400:12345-192.168.203.43@tcp:103:md5sum.0 Request processed in 144us (327us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784716.991284:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 34 00000100:00100000:0.0:1787784716.996312:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784716.996972:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658054528 00000100:00100000:1.0:1787784716.996982:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658054528 00000100:00100000:1.0:1787784716.997042:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 850 00000100:00100000:1.0:1787784716.997053:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:24645:x1874627658054528:12345-192.168.203.43@tcp:58:md5sum.0 00010000:00010000:1.0:1787784716.997188:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.997204:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784716.997937:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787784716.998584:0:5811:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].21, 136 off11048 00000100:00100000:1.0:1787784716.998720:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:24645:x1874627658054528:12345-192.168.203.43@tcp:58:md5sum.0 Request processed in 1666us (2416us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784716.998730:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 850 00000100:00100000:3.0:1787784717.001617:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.001735:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658054656 00000100:00100000:2.0:1787784717.001744:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658054656 00000100:00100000:2.0:1787784717.001806:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 851 00000100:00100000:2.0:1787784717.001814:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:24645:x1874627658054656:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:2.0:1787784717.001845:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784717.001865:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bb4 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.001918:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787784717.001996:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784717.002024:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 4/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.002041:0:10850:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.002055:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 3/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.002083:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:2.0:1787784717.002086:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784717.002090:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 3/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.002097:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784717.002520:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:24645:x1874627658054656:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 529us (733us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.002530:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 851 00010000:00010000:3.0:1787784717.233512:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784717.745991:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784717.746001:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784717.746033:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 22 count 4 00000040:00080000:2.0:1787784717.746050:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784717.746202:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668376704:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784717.746292:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063efd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784717.754451:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784717.754595:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658054784 00000100:00100000:0.0:1787784717.754613:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658054784 00000100:00100000:0.0:1787784717.754704:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 852 00000100:00100000:0.0:1787784717.754714:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658054784:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784717.754751:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784717.754851:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658054784:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 138us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784717.754860:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 852 00000100:00080000:1.0:1787784717.755077:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784717.756604:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668376704:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784717.760461:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784717.760602:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658054912 00000100:00100000:1.0:1787784717.760613:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658054912 00000100:00100000:1.0:1787784717.760679:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 853 00000100:00100000:1.0:1787784717.760689:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24647:x1874627658054912:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1787784717.761241:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761260:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761273:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761286:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761293:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761298:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761319:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761326:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761332:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784717.761368:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00000004:00200000:1.0:1787784717.761383:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784717.761397:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761408:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761423:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761430:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761436:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784717.761440:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761447:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761460:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784717.761463:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784717.761468:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761474:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5454 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761603:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761611:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761617:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761622:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784717.761632:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761639:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761647:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xb1925f2 (ffff989531111b00) 00010000:00010000:1.0:1787784717.761650:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784717.761653:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761659:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc545b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761674:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761679:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761684:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761689:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784717.761691:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761696:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761703:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:1.0:1787784717.761705:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784717.761714:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.761720:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5462 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784717.761815:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24647:x1874627658054912:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 1125us (1362us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784717.761824:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 853 00000100:00100000:0.0:1787784717.766393:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.766447:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658055168 00000100:00100000:2.0:1787784717.766457:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658055168 00000100:00100000:2.0:1787784717.766529:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 854 00000100:00100000:2.0:1787784717.766540:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508340450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:24647:x1874627658055168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784717.766560:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784717.766579:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5469 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bbb expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766673:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766687:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766698:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784717.766710:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:2.0:1787784717.766816:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766824:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766830:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766852:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784717.766867:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766876:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766889:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:2.0:1787784717.766893:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784717.766898:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.766904:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5470 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.767244:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5469 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bbb expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.767257:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784717.767259:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784717.767264:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5469 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bbb expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.767274:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784717.767340:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508340450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658055168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 801us (955us total) trans 4294967865 rc 301/301 00000100:00100000:2.0:1787784717.767348:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 854 00000100:00100000:3.0:1787784717.777653:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784717.777829:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658055296 00000100:00100000:1.0:1787784717.777839:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658055296 00000100:00100000:1.0:1787784717.778004:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 855 00000100:00100000:1.0:1787784717.778017:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658055296:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784717.778056:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784717.778199:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658055296:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 180us (551us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784717.778207:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 855 00000100:00100000:0.0:1787784717.785839:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.786637:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658055424 00000100:00100000:3.0:1787784717.786654:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658055424 00000100:00100000:3.0:1787784717.786721:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 856 00000100:00100000:3.0:1787784717.786731:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658055424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784717.786762:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784717.786787:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc547e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bc2 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.787021:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.787036:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.787053:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784717.787238:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784717.787242:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 23 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784717.787270:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.787294:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc547e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 40 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.787311:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.787320:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:3.0:1787784717.787323:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784717.787326:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135000, rc 0) 00000100:00100000:3.0:1787784717.788673:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:24647:x1874627658055424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1935us (2837us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.788685:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 856 00000100:00100000:0.0:1787784717.795560:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.796119:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658055552 00000100:00100000:2.0:1787784717.796179:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658055552 00000100:00100000:2.0:1787784717.796233:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 857 00000100:00100000:2.0:1787784717.796243:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+39:24647:x1874627658055552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784717.796263:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784717.796282:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc548c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bc9 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796349:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796364:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796386:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796401:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796408:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796415:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796534:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796543:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796550:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784717.796592:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 512 00000004:00200000:2.0:1787784717.796596:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784717.796619:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796631:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796638:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796644:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796650:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784717.796654:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796661:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796673:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784717.796676:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784717.796681:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796686:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5493 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796699:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796715:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796720:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.796730:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fda00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784717.796733:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.797361:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.797559:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff989530cb1c00) 00010000:00010000:2.0:1787784717.797562:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784717.797566:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.797573:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc549a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.797614:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc548c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.797627:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 40 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.797635:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784717.797637:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784717.797649:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784717.797717:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:24647:x1874627658055552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1475us (2170us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.797725:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 857 00000100:00100000:1.0:1787784717.810012:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668376832:0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.810092:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784717.810275:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668377216:0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.810303:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784717.810328:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784717.810333:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668377088:0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.810353:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784717.810627:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668377344:0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.810651:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784717.810667:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784717.810683:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a92d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668376960:0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.810703:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784717.810725:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668377088 00000100:00100000:1.0:1787784717.810731:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668377088 00000100:00100000:3.0:1787784717.810835:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668376832 00000100:00100000:3.0:1787784717.810839:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668376832 00000100:00100000:3.0:1787784717.810887:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:3.0:1787784717.810895:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668376832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784717.811036:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668376832:12345-0@lo:400:kworker.0 Request processed in 142us (944us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.811044:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00100000:3.0:1787784717.811102:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668376832:0@lo:400:kworker.0 00000100:00100000:3.0:1787784717.811120:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668377344 00000100:00100000:3.0:1787784717.811129:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668377344 00000100:00100000:3.0:1787784717.811178:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668376960 00000100:00100000:3.0:1787784717.811181:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668376960 00000100:00100000:3.0:1787784717.811193:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:3.0:1787784717.811198:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668377344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784717.811230:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668377344:12345-0@lo:400:kworker.0 Request processed in 32us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.811236:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:3.0:1787784717.811246:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:3.0:1787784717.811251:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668376960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784717.811270:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668376960:12345-0@lo:400:kworker.0 Request processed in 19us (566us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.811275:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00080000:3.0:1787784717.811287:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784717.811303:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a92d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668376960:0@lo:400:kworker.0 00000100:00100000:3.0:1787784717.811311:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668377344:0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.811670:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1787784717.811677:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668377088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.811713:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668377088:12345-0@lo:400:kworker.0 Request processed in 36us (1359us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784717.811718:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:0.0:1787784717.811839:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668377216 00000100:00100000:0.0:1787784717.811843:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668377216 00000100:00100000:0.0:1787784717.811883:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:0.0:1787784717.811890:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668377216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784717.812181:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668377216:12345-0@lo:400:kworker.0 Request processed in 292us (1877us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784717.812188:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:0.0:1787784717.812336:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668377472:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784717.812367:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784717.812437:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668377088:0@lo:400:kworker.0 00000100:00100000:1.0:1787784717.812451:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668377216:0@lo:400:kworker.0 00000100:00080000:3.0:1787784717.812866:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784717.812872:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784717.815933:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.816187:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658055808 00000100:00100000:3.0:1787784717.816196:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658055808 00000100:00100000:3.0:1787784717.816259:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 858 00000100:00100000:3.0:1787784717.816269:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:24647:x1874627658055808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784717.816290:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784717.816310:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc54a8 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bd0 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784717.817557:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668377472 00000100:00100000:1.0:1787784717.817567:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668377472 00000100:00100000:1.0:1787784717.817617:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:1.0:1787784717.817627:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668377472:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784717.817664:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 81592320 pending 8519680 free 3857711104 avail 3855613952 00000100:00080000:3.0:1787784717.817754:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787784717.819437:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc54af lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.819455:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc54af lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.819467:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc54af lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784717.819478:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 0 00010000:00010000:2.0:1787784717.819929:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc54a8 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd0 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.819953:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc54af lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd0 expref: 42 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.819961:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:2.0:1787784717.819964:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784717.819968:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784717.820041:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:24647:x1874627658055808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3773us (4117us total) trans 4294967866 rc 0/0 00000100:00100000:2.0:1787784717.820050:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 858 00000100:00100000:1.0:1787784717.822565:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668377472:12345-0@lo:13:osp-pre-1-0.0 Request processed in 4933us (10189us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784717.822579:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:0.0:1787784717.823991:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668377472:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784717.829697:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784717.829876:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658056064 00000100:00100000:1.0:1787784717.829886:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658056064 00000100:00100000:1.0:1787784717.829942:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 859 00000100:00100000:1.0:1787784717.829953:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:24647:x1874627658056064:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784717.830010:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784717.830037:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc54af lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd0 expref: 42 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.830081:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:1.0:1787784717.830092:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784717.830115:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.830129:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.830138:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 4/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.830229:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784717.830244:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 102 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784717.830259:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668377600:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784717.830331:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ebe40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784717.833124:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668377600:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784717.833169:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784717.835006:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.835358:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658056192 00000100:00100000:2.0:1787784717.835376:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658056192 00010000:00010000:2.0:1787784717.835418:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 103 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784717.835480:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 265 00000100:00100000:2.0:1787784717.835490:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ee850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:2357:x1874627658056192:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784717.835507:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784717.835513:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 103 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.835526:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 103 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.835549:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc53b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b83 expref: 103 pid: 5811 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.835563:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784717.835569:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.835577:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.835584:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.835615:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784717.835663:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ee850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:2357:x1874627658056192:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 174us (664us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.835671:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 265 00010000:00010000:3.0:1787784717.835988:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.835997:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836020:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836029:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836037:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836119:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836126:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836132:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836137:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784717.836141:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836188:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836210:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784717.836212:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784717.836216:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836222:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc54bd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836231:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836237:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836242:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836245:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784717.836248:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836253:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836258:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x2ab11b48 (ffff989505542f00) 00010000:00010000:3.0:1787784717.836260:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784717.836263:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.836268:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54c4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x2ab11b48 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784717.836338:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:24647:x1874627658056064:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 6381us (6644us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784717.836345:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 859 00000100:00100000:3.0:1787784717.840616:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.840683:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658056320 00000100:00100000:2.0:1787784717.840707:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658056320 00000100:00100000:2.0:1787784717.840768:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 860 00000100:00100000:2.0:1787784717.840779:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:24647:x1874627658056320:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784717.840798:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784717.840822:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54cb lrc: 2/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bd7 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.840856:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.840871:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.840883:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.840976:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54cb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 43 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.840995:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 42 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.841004:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:2.0:1787784717.841007:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784717.841010:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784717.841086:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:24647:x1874627658056320:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 308us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.841094:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 860 00000100:00100000:0.0:1787784717.844809:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.844882:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658056576 00000100:00100000:3.0:1787784717.844892:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658056576 00000100:00100000:3.0:1787784717.844954:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 861 00000100:00100000:3.0:1787784717.844964:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:24647:x1874627658056576:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784717.845042:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.845073:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.845085:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784717.845309:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000962dbd1f reply state 000000007df9d9dd transno 4294967867 ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784717.845319:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784717.845543:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:24647:x1874627658056576:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 580us (744us total) trans 4294967867 rc 0/0 00000100:00100000:3.0:1787784717.845551:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 861 00010000:00010000:0.0:1787784717.848000:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784717.848017:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784717.848026:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784717.848048:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784717.848068:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784717.848076:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784717.848093:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:0.0:1787784717.848096:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784717.848100:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784717.848106:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc54d9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784717.850814:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.858294:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658056704 00000100:00100000:2.0:1787784717.858326:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658056704 00000100:00100000:2.0:1787784717.858396:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 862 00000100:00100000:2.0:1787784717.858407:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:24647:x1874627658056704:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784717.858431:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784717.858452:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bde expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.858512:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.858527:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.858537:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:2.0:1787784717.858540:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784717.858545:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fda00, rc 0) 00000100:00100000:2.0:1787784717.858627:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:24647:x1874627658056704:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 221us (7822us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.858634:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 862 00000100:00100000:1.0:1787784717.866264:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.866488:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658056832 00000100:00100000:3.0:1787784717.866497:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658056832 00000100:00100000:3.0:1787784717.866551:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 175 00000100:00100000:3.0:1787784717.866562:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:24647:x1874627658056832:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1787784717.866701:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:24647:x1874627658056832:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 140us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.866709:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 175 00000100:00100000:1.0:1787784717.872358:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.872546:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658057088 00000100:00100000:3.0:1787784717.872556:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658057088 00000100:00100000:3.0:1787784717.872609:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 176 00000100:00100000:3.0:1787784717.872619:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:24647:x1874627658057088:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784717.872850:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:24647:x1874627658057088:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 231us (500us total) trans 4294967868 rc 0/0 00000100:00100000:3.0:1787784717.872858:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 176 00000100:00100000:0.0:1787784717.876218:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.876305:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658057344 00000100:00100000:3.0:1787784717.876315:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658057344 00000100:00100000:3.0:1787784717.876366:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 863 00000100:00100000:3.0:1787784717.876387:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:24647:x1874627658057344:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784717.876501:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:24647:x1874627658057344:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 115us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.876509:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 863 00000100:00100000:0.0:1787784717.883098:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.883893:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658057600 00000100:00100000:2.0:1787784717.883904:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658057600 00000100:00100000:2.0:1787784717.883964:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 864 00000100:00100000:2.0:1787784717.883974:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:24647:x1874627658057600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784717.883994:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784717.884014:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc54e7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0be5 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.884241:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.884256:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.884265:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 4/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.884298:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.884313:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 46 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784717.884328:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668377728:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784717.884654:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784717.890638:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668377728:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784717.890669:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784717.894361:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.894633:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658057728 00000100:00100000:2.0:1787784717.894652:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658057728 00010000:00010000:2.0:1787784717.894700:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 47 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784717.894742:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 266 00000100:00100000:2.0:1787784717.894765:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658057728:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784717.894781:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784717.894788:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 47 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.894842:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 47 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.894865:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc54e0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bde expref: 47 pid: 5812 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.894883:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:2.0:1787784717.894888:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.894897:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.894906:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.894935:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784717.894989:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2356:x1874627658057728:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 225us (637us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.895006:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 266 00010000:00010000:3.0:1787784717.895394:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895403:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784717.895431:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 0 00000004:00080000:3.0:1787784717.895614:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x11:0x0], index 0: 0 00010000:00010000:3.0:1787784717.895768:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895784:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895809:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895815:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784717.895819:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895826:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895838:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:3.0:1787784717.895841:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784717.895845:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 2/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895851:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd400/0xb0ce63137acc54ee lrc: 0/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895898:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc54e7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0be5 expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895908:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784717.895911:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784717.895914:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc54e7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0be5 expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.895923:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:3.0:1787784717.895969:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:24647:x1874627658057600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 11996us (12881us total) trans 4294967869 rc 301/301 00000100:00100000:3.0:1787784717.895977:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 864 00000100:00100000:0.0:1787784717.901549:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.904226:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658057984 00000100:00100000:3.0:1787784717.904241:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658057984 00000100:00100000:3.0:1787784717.904471:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:3.0:1787784717.904482:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:24647:x1874627658057984:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784717.906481:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:24647:x1874627658057984:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1992us (4995us total) trans 4294967870 rc 0/0 00000100:00100000:3.0:1787784717.906563:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 177 00000100:00100000:1.0:1787784717.910023:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.910478:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658058112 00000100:00100000:3.0:1787784717.910488:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658058112 00000100:00100000:3.0:1787784717.911222:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 865 00000100:00100000:3.0:1787784717.911236:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:24647:x1874627658058112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784717.911261:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784717.911287:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54fc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bec expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.911357:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc5503 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.911371:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc5503 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.911383:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc5503 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784717.911426:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 0 00000004:00200000:3.0:1787784717.911443:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784717.911464:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc5503 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.911491:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc54fc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bec expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.911510:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc5503 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bec expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.911518:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:3.0:1787784717.911521:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1787784717.911525:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135e00, rc 0) 00000100:00100000:3.0:1787784717.911597:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:24647:x1874627658058112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 365us (1583us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.911604:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 865 00000100:00100000:0.0:1787784717.914977:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.915334:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658058240 00000100:00100000:2.0:1787784717.915344:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658058240 00000100:00100000:2.0:1787784717.915398:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 214 00000100:00100000:2.0:1787784717.915423:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658058240:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784717.915456:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784717.915478:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bf3 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784717.915503:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784717.915521:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=509s v=(0 0) 00010000:00010000:2.0:1787784717.915525:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784717.915535:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.915554:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.915772:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fda00, rc 0) 00000100:00100000:2.0:1787784717.915876:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658058240:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 468us (908us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.915884:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 214 00000100:00100000:0.0:1787784717.926881:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.926978:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658058496 00000100:00100000:2.0:1787784717.926988:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658058496 00000100:00100000:2.0:1787784717.927082:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 866 00000100:00100000:2.0:1787784717.927092:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:24647:x1874627658058496:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784717.927240:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:24647:x1874627658058496:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 149us (369us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.927247:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 866 00000100:00100000:1.0:1787784717.945858:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784717.946101:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658058624 00000100:00100000:3.0:1787784717.946111:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658058624 00000100:00100000:3.0:1787784717.946198:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 215 00000100:00100000:3.0:1787784717.946208:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:24647:x1874627658058624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784717.946225:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784717.946243:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0bfa expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.946273:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.946281:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784717.946301:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784717.946307:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.946363:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.946376:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 10 pid: 10688 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784717.946389:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b10780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668377856:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784717.946461:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7ae240 going to sleep for 11 seconds 00000100:00100000:3.0:1787784717.950095:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b10780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668377856:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784717.950131:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784717.950173:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539134a00, rc 0) 00000100:00100000:3.0:1787784717.950232:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:24647:x1874627658058624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 4025us (4384us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784717.950241:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 215 00000100:00100000:1.0:1787784717.953986:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784717.954401:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658058752 00000100:00100000:2.0:1787784717.954426:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658058752 00010000:00010000:2.0:1787784717.954466:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 10 pid: 10688 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784717.954508:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 267 00000100:00100000:2.0:1787784717.954518:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658058752:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784717.954532:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784717.954552:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 10 pid: 10688 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.954566:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 10 pid: 10688 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.954588:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bf7da85/0xb0ce63137acc550a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bf3 expref: 10 pid: 10688 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.954605:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784717.954610:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784717.954617:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.954628:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784717.954684:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784717.954710:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668377984:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784717.954791:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784717.961472:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668377984:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784717.961499:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784717.961558:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658058752:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 7041us (7581us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784717.961570:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 267 00000100:00100000:0.0:1787784717.995228:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784717.995517:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658058880 00000100:00100000:1.0:1787784717.995527:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658058880 00002000:00100000:1.0:1787784717.995578:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x11:0x0] (0->1048575) req@ffff989402a93480 x1874627658058880/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:598/0 lens 488/0 e 0 to 0 dl 1787784728 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784717.995589:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402a93480 with x1874627658058880 ext(0->1048575) 00002000:00010000:1.0:1787784717.995594:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402a93480 00000100:00100000:1.0:1787784717.995627:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 75 00000100:00100000:1.0:1787784717.995636:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658058880:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1787784718.044574:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.044661:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059008 00000100:00100000:3.0:1787784718.044679:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059008 00002000:00100000:3.0:1787784718.044745:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x11:0x0] (1048576->2097151) req@ffff98953f09e940 x1874627658059008/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784718.044756:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f09e940 with x1874627658059008 ext(1048576->2097151) 00002000:00010000:3.0:1787784718.044773:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98953f09e940 00000100:00100000:3.0:1787784718.044808:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 76 00000100:00100000:3.0:1787784718.044818:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658059008:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00080000:0.0:1787784718.045023:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784718.050206:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658058880:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 54560us (54977us total) trans 4294967333 rc 0/0 00000100:00100000:2.0:1787784718.050220:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 75 00002000:00100000:2.0:1787784718.050237:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x11:0x0] (0->1048575) req@ffff989402a93480 x1874627658058880/t4294967333(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:598/0 lens 488/448 e 0 to 0 dl 1787784728 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784718.050247:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402a93480 with x1874627658058880 ext(0->1048575) 00002000:00010000:2.0:1787784718.050253:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402a93480 00000100:00100000:0.0:1787784718.080603:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658059008:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 35775us (36027us total) trans 4294967334 rc 0/0 00000100:00100000:0.0:1787784718.080618:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 76 00002000:00100000:0.0:1787784718.080631:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x11:0x0] (1048576->2097151) req@ffff98953f09e940 x1874627658059008/t4294967334(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/448 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784718.080664:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f09e940 with x1874627658059008 ext(1048576->2097151) 00002000:00010000:0.0:1787784718.080671:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98953f09e940 00000100:00100000:3.0:1787784718.083196:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784718.083364:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059136 00000100:00100000:2.0:1787784718.083373:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059136 00002000:00100000:2.0:1787784718.083418:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (2097152->3145727) req@ffff989402b10780 x1874627658059136/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784718.083428:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402b10780 with x1874627658059136 ext(2097152->3145727) 00002000:00010000:2.0:1787784718.083433:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402b10780 00000100:00100000:2.0:1787784718.083456:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 77 00000100:00100000:2.0:1787784718.083464:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2354:x1874627658059136:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1787784718.112967:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784718.113085:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059264 00000100:00100000:2.0:1787784718.113096:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059264 00002000:00100000:2.0:1787784718.113182:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x11:0x0] (3145728->4194303) req@ffff989402b130c0 x1874627658059264/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784718.113195:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402b130c0 with x1874627658059264 ext(3145728->4194303) 00002000:00010000:2.0:1787784718.113200:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402b130c0 00000100:00100000:2.0:1787784718.113224:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 78 00000100:00100000:2.0:1787784718.113235:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2354:x1874627658059264:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1787784718.127884:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784718.127971:0:6674:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 4 min 3 max 116 00000100:00100000:0.0:1787784718.127983:0:6674:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_004' 00000100:00100000:0.0:1787784718.128249:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059392 00000100:00100000:0.0:1787784718.128258:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059392 00002000:00100000:0.0:1787784718.128300:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x11:0x0] (4194304->5242879) req@ffff989403126d00 x1874627658059392/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784718.128310:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403126d00 with x1874627658059392 ext(4194304->5242879) 00002000:00010000:0.0:1787784718.128315:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989403126d00 00000100:00100000:2.0:1787784718.128849:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 79 00000100:00100000:2.0:1787784718.128861:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658059392:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1787784718.149292:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2354:x1874627658059136:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 65817us (66095us total) trans 4294967335 rc 0/0 00000100:00100000:3.0:1787784718.149307:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 77 00002000:00100000:3.0:1787784718.149320:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (2097152->3145727) req@ffff989402b10780 x1874627658059136/t4294967335(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/448 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784718.149331:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402b10780 with x1874627658059136 ext(2097152->3145727) 00002000:00010000:3.0:1787784718.149336:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402b10780 00000100:00100000:3.0:1787784718.162868:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2354:x1874627658059264:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 49622us (49896us total) trans 4294967336 rc 0/0 00000100:00100000:3.0:1787784718.162884:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 78 00002000:00100000:3.0:1787784718.162899:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x11:0x0] (3145728->4194303) req@ffff989402b130c0 x1874627658059264/t4294967336(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/448 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784718.162909:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402b130c0 with x1874627658059264 ext(3145728->4194303) 00002000:00010000:3.0:1787784718.162915:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402b130c0 00000100:00100000:3.0:1787784718.167853:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2357:x1874627658059392:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 38985us (39969us total) trans 4294967337 rc 0/0 00000100:00100000:3.0:1787784718.167868:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 79 00002000:00100000:3.0:1787784718.167881:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x11:0x0] (4194304->5242879) req@ffff989403126d00 x1874627658059392/t4294967337(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:599/0 lens 488/448 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784718.167911:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403126d00 with x1874627658059392 ext(4194304->5242879) 00002000:00010000:3.0:1787784718.167917:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989403126d00 00000100:00100000:0.0:1787784718.172520:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784718.172716:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059520 00000100:00100000:2.0:1787784718.172726:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059520 00000100:00100000:2.0:1787784718.172819:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 867 00000100:00100000:2.0:1787784718.172829:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:24647:x1874627658059520:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1787784718.205225:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967864 is committed 00000001:00080000:3.0:1787784718.205244:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967853 is committed 00000020:00080000:3.0:1787784718.205255:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 12 batchid updated 00000004:00080000:3.0:1787784718.205274:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 12 committed 00000020:00080000:3.0:1787784718.205290:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98954066db40 refcount 2 committed 0 result 0 batchid 13 00000020:00080000:3.0:1787784718.205295:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989540421c00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98954066d900 00000020:00080000:3.0:1787784718.205299:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895188ac700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989402f15400 00000001:00080000:3.0:1787784718.205316:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967866 is committed 00000004:00080000:3.0:1787784718.205337:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 8 committed 00000004:00080000:3.0:1787784718.205345:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 11 committed 00000001:00080000:3.0:1787784718.205354:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967870 is committed 00000100:00100000:3.0:1787784718.205543:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668378112:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1787784718.205603:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784718.205673:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784718.205680:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668378240:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1787784718.205702:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784718.205728:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b11680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668378368:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784718.205751:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784718.205791:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668378112 00000100:00100000:3.0:1787784718.205799:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668378112 00000100:00100000:3.0:1787784718.205838:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668378240 00000100:00100000:3.0:1787784718.205841:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668378240 00000100:00100000:3.0:1787784718.205856:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:3.0:1787784718.205864:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+8:3278:x1874627668378112:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1787784718.205902:0:6670:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:18 count 1 00010000:00010000:3.0:1787784718.205926:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.205947:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 21 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.205957:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=3s v=(0 0) 00010000:00010000:3.0:1787784718.205962:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.205966:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.205993:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 21 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206007:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 21 pid: 6671 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.206018:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668378496:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784718.206087:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7ae240 going to sleep for 11 seconds 00000100:00100000:3.0:1787784718.206102:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668378368 00000100:00100000:3.0:1787784718.206105:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668378368 00000100:00100000:3.0:1787784718.206126:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1787784718.206133:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3278:x1874627668378240:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1787784718.206143:0:6669:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:19 count 1 00002000:00010000:3.0:1787784718.206202:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x13:0x0] initial LVB size: 5242880, mtime: 0x6a8f6e0b, atime: 0x0, ctime: 0x6a8f6e0b, blocks: 0x1 00010000:00010000:3.0:1787784718.206214:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 3/0,1 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206222:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=509s v=(0 0) 00010000:00010000:3.0:1787784718.206225:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.206230:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206245:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206254:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206261:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206279:0:6669:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784718.206593:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3278:x1874627668378240:12345-0@lo:6:osp-syn-1-0.0 Request processed in 459us (888us total) trans 4294967348 rc 0/0 00000100:00100000:3.0:1787784718.206601:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1787784718.206621:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1787784718.206628:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668378368:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787784718.206642:0:6669:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:16 count 1 00010000:00010000:3.0:1787784718.206657:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206682:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206688:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=3s v=(0 0) 00010000:00010000:3.0:1787784718.206691:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.206694:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206714:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206723:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 11 pid: 13825 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.206735:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b10000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668378624:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784718.206773:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503a20b40 going to sleep for 11 seconds 00010000:00010000:3.0:1787784718.206801:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784718.206807:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206814:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784718.206818:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206826:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206847:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 1/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206854:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042efad76/0xb0ce63137acc551f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.206872:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784718.206874:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784718.206876:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1787784718.206907:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402b13840/2, rc 0, transno 4294967348 00000100:00100000:3.0:1787784718.206914:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668378240:0@lo:6:osp-syn-1-0.0 00000020:00080000:0.0:1787784718.210519:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 13 committed batchid 12 00000020:00080000:0.0:1787784718.210755:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 12 00000100:00100000:2.0:1787784718.211735:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658059520:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 38900us (39216us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784718.211747:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 867 00000100:00100000:3.0:1787784718.212041:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.212098:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.212122:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059648 00000100:00100000:3.0:1787784718.212128:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059648 00010000:00010000:3.0:1787784718.212195:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 22 pid: 6671 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.212220:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059776 00000100:00100000:3.0:1787784718.212223:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059776 00010000:00010000:3.0:1787784718.212241:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 12 pid: 13825 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.212261:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 268 00000100:00100000:3.0:1787784718.212269:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+22:2356:x1874627658059648:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784718.212299:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784718.212305:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 22 pid: 6671 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212318:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 22 pid: 6671 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212346:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.212350:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.212356:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212365:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212386:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784718.212415:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+22:2356:x1874627658059648:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 146us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784718.212437:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 268 00000100:00100000:3.0:1787784718.212457:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 269 00000100:00100000:3.0:1787784718.212465:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658059776:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784718.212474:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784718.212479:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 12 pid: 13825 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212504:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 12 pid: 13825 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212519:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.212522:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.212526:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212533:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212548:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784718.212582:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658059776:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 118us (483us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784718.212588:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 269 00000100:00080000:3.0:1787784718.212608:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784718.212611:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787784718.212697:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc5343 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b67 expref: 11 pid: 13825 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.212712:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b10000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668378624:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784718.212733:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212743:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212756:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212771:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212778:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.212796:0:6669:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784718.213371:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668378368:12345-0@lo:6:osp-syn-0-0.0 Request processed in 6741us (7617us total) trans 4294967338 rc 0/0 00000100:00100000:3.0:1787784718.213381:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00010000:00010000:3.0:1787784718.213408:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000004:00080000:1.0:1787784718.213414:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402b11680/2, rc 0, transno 4294967338 00010000:00010000:3.0:1787784718.213415:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784718.213425:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b11680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668378368:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1787784718.213428:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784718.213434:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.213444:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.213459:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.213474:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023f7dba4/0xb0ce63137acc5526 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.213492:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784718.213494:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784718.213495:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784718.213768:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e175fd38/0xb0ce63137acc534a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b60 expref: 21 pid: 6671 timeout: 609 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784718.213779:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668378496:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784718.213793:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.213880:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.213888:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.213898:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.213906:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.213933:0:6670:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787784718.214319:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+9:3278:x1874627668378112:12345-0@lo:6:osp-syn-1-0.0 Request processed in 8454us (8714us total) trans 4294967349 rc 0/0 00000100:00100000:2.0:1787784718.214328:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000004:00080000:1.0:1787784718.214731:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402b121c0/2, rc 0, transno 4294967349 00000100:00100000:1.0:1787784718.214740:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668378112:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1787784718.217051:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784718.217526:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.217540:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784718.217545:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.217557:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.217579:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.217587:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a7c0d90/0xb0ce63137acc5518 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.217609:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784718.217611:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784718.217613:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784718.218542:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.218635:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658059904 00000100:00100000:3.0:1787784718.218784:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658059904 00000100:00100000:3.0:1787784718.218947:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 219 00000100:00100000:3.0:1787784718.218960:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2356:x1874627658059904:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00010000:00010000:3.0:1787784718.258491:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784718.273703:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967334 is committed 00000001:00080000:0.0:1787784718.273732:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967338 is committed 00000001:00080000:0.0:1787784718.273737:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967337 is committed 00000100:00100000:2.0:1787784718.274236:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658059904:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 55271us (55694us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784718.274249:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 219 00000100:00100000:0.0:1787784718.278563:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.278985:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658060160 00000100:00100000:3.0:1787784718.278995:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658060160 00000100:00100000:2.0:1787784718.279245:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 868 00000100:00100000:0.0:1787784718.279320:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658060160:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784718.279373:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784718.279402:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc5503 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bec expref: 38 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.279422:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:0.0:1787784718.279428:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784718.279459:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.279486:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.279496:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784718.279695:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000071baefcf reply state 0000000022f0415c transno 4294967871 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784718.279705:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784718.279805:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+39:24647:x1874627658060160:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 550us (1251us total) trans 4294967871 rc 0/0 00000100:00100000:0.0:1787784718.284537:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 868 00000100:00100000:1.0:1787784718.286683:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.286900:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658060288 00000100:00100000:3.0:1787784718.286910:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658060288 00000100:00100000:3.0:1787784718.286964:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 869 00000100:00100000:3.0:1787784718.286974:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:24647:x1874627658060288:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784718.286996:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784718.287015:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c01 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.287086:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 40 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.287099:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 40 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.287116:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:3.0:1787784718.287120:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784718.287123:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784718.287198:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:24647:x1874627658060288:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 226us (523us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784718.287206:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 869 00010000:00010000:0.0:1787784718.288974:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.288990:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.288998:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.289004:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e2bc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784718.289009:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.289017:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.289034:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:0.0:1787784718.289753:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784718.289760:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.289768:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e2bc00/0xb0ce63137acc552d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784718.290897:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.291034:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658060416 00000100:00100000:3.0:1787784718.291070:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658060416 00000100:00100000:3.0:1787784718.291127:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 220 00000100:00100000:3.0:1787784718.291138:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658060416:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1787784718.291417:0:6671:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 0 -> 5242880 00002000:00010000:3.0:1787784718.291421:0:6671:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb mtime from disk: 0 -> 1787784715 00002000:00010000:3.0:1787784718.291424:0:6671:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb atime from disk: 0 -> 1787784715 00002000:00010000:3.0:1787784718.291426:0:6671:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb ctime from disk: 0 -> 1787784715 00002000:00010000:3.0:1787784718.291429:0:6671:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 1 -> 10247 00000100:00100000:3.0:1787784718.291480:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658060416:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 343us (591us total) trans 4294967339 rc 0/0 00000100:00100000:3.0:1787784718.291488:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 220 00000100:00100000:1.0:1787784718.294928:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.295092:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658060672 00000100:00100000:3.0:1787784718.295101:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658060672 00000100:00100000:3.0:1787784718.295189:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 870 00000100:00100000:3.0:1787784718.295199:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:24647:x1874627658060672:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1787784718.295261:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.295277:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.295286:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.295319:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.295333:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 39 pid: 5812 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.295363:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668378752:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784718.295433:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f097600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784718.298430:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668378752:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784718.298458:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784718.299791:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784718.300108:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658060800 00000100:00100000:0.0:1787784718.300118:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658060800 00010000:00010000:0.0:1787784718.300202:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 40 pid: 5812 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784718.300245:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 270 00000100:00100000:0.0:1787784718.300255:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2356:x1874627658060800:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784718.300271:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784718.300278:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 40 pid: 5812 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.300292:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 40 pid: 5812 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.300316:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc54d2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bd7 expref: 40 pid: 5812 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.300334:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:0.0:1787784718.300339:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.300348:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.300369:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.300405:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784718.300455:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2356:x1874627658060800:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 201us (671us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784718.300463:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 270 00010000:00010000:2.0:1787784718.300538:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300546:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300582:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300589:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300595:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300601:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784718.300605:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300612:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300621:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:2.0:1787784718.300624:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787784718.300627:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.300633:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc553b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784718.300669:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658060672:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 5472us (5752us total) trans 0 rc -22/-22 00000100:00100000:2.0:1787784718.300678:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 870 00000100:00100000:1.0:1787784718.304084:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784718.304193:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658060928 00000100:00100000:2.0:1787784718.304203:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658060928 00000100:00100000:2.0:1787784718.304264:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 221 00000100:00100000:2.0:1787784718.304286:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658060928:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784718.304333:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.304352:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.304361:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784718.304366:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784718.304370:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.304403:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.304417:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 10 pid: 6671 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784718.304431:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668378880:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784718.304499:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506773b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784718.311576:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668378880:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784718.311608:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784718.318118:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.318973:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658061056 00000100:00100000:3.0:1787784718.318985:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658061056 00010000:00010000:3.0:1787784718.319028:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 11 pid: 6671 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.319413:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:3.0:1787784718.319425:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658061056:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784718.319442:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784718.319465:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 11 pid: 6671 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.319479:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 11 pid: 6671 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.319501:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b207fcc/0xb0ce63137acc5511 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bfa expref: 11 pid: 6671 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.319518:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784718.319537:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784718.319543:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.319553:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.319629:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784718.319708:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658061056:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 285us (1600us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784718.319718:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 271 00010000:00010000:0.0:1787784718.320787:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320802:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320831:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320840:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320849:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320923:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784718.320929:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320955:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320970:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.320979:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057f246d1/0xb0ce63137acc5542 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784718.321118:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658060928:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 16833us (17042us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784718.321127:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 221 00000100:00100000:1.0:1787784718.325529:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.325818:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658061312 00000100:00100000:3.0:1787784718.325829:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658061312 00000100:00100000:3.0:1787784718.325987:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 871 00000100:00100000:3.0:1787784718.325998:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:24647:x1874627658061312:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784718.326093:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.326111:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.326124:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.326395:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.326408:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.326427:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.326462:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.326476:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 39 pid: 5811 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.326490:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668379008:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784718.326558:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f097600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784718.330095:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784718.330175:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658061440 00000100:00100000:0.0:1787784718.330184:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658061440 00010000:00010000:0.0:1787784718.330220:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 40 pid: 5811 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784718.330258:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 272 00000100:00100000:0.0:1787784718.330266:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ed450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2357:x1874627658061440:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784718.330280:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784718.330286:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 40 pid: 5811 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.330299:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 40 pid: 5811 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.330337:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:0.0:1787784718.330343:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.330351:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.330358:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784718.330374:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784718.330456:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ed450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2357:x1874627658061440:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 190us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784718.330463:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 272 00010000:00010000:3.0:1787784718.331137:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5534 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c01 expref: 39 pid: 5811 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.331233:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668379008:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784718.331252:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.331261:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.331268:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784718.331480:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967872 00000001:00080000:3.0:1787784718.331602:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967873 00000001:00080000:3.0:1787784718.331637:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967874 00000100:00100000:3.0:1787784718.331801:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5811:1874627668379136:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787784718.331851:0:5811:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784718.331982:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f097b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784718.333506:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668379136 00000100:00100000:0.0:1787784718.333518:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668379136 00000100:00100000:0.0:1787784718.333624:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 31 00000100:00100000:0.0:1787784718.333634:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5811:x1874627668379136:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787784718.333771:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5811:x1874627668379136:12345-0@lo:1000:mdt00_000.0 Request processed in 138us (1919us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784718.333779:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 31 00000100:00100000:3.0:1787784718.333846:0:5811:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967338 gen 2 00000004:00080000:3.0:1787784718.333852:0:5811:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402b11680, transno 4294967338 00000100:00100000:3.0:1787784718.333879:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5811:1874627668379136:0@lo:1000:mdt00_000.0 00000001:00080000:3.0:1787784718.334439:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967875 00000100:00100000:3.0:1787784718.334513:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5811:1874627668379264:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787784718.334553:0:5811:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784718.335564:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668379264 00000100:00100000:1.0:1787784718.335575:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668379264 00000100:00100000:1.0:1787784718.335615:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 32 00000100:00100000:1.0:1787784718.335632:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5811:x1874627668379264:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787784718.335875:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5811:x1874627668379264:12345-0@lo:1000:mdt00_000.0 Request processed in 244us (1321us total) trans 4294967340 rc 0/0 00000100:00100000:1.0:1787784718.335883:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 32 00000100:00100000:3.0:1787784718.336241:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f097600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784718.336275:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5811:1874627668379264:0@lo:1000:mdt00_000.0 00000001:00080000:3.0:1787784718.336369:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967876 00010000:00010000:3.0:1787784718.336402:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336412:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336420:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336426:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784718.336430:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336437:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336452:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:3.0:1787784718.336463:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784718.336469:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336475:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5550 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336507:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336513:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336519:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336523:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784718.336526:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336531:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336539:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:3.0:1787784718.336541:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784718.336547:0:5811:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 81 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336559:0:5811:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 81 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336568:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 81 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.336603:0:5811:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 81 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.336615:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402187480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668379392:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784718.336745:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f097600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784718.343481:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.343586:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402187480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668379392:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784718.343614:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343625:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5446 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343657:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343671:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343682:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343692:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343699:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343705:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343726:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784718.343730:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343738:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343751:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:3.0:1787784718.343754:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784718.343757:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343762:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5557 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343772:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343778:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343784:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343788:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784718.343791:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343798:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343816:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:3.0:1787784718.343819:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784718.343822:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.343827:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5549 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784718.344202:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:24647:x1874627658061312:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 18204us (18680us total) trans 4294967877 rc 0/0 00000100:00100000:3.0:1787784718.344213:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 871 00000020:00080000:3.0:1787784718.344220:0:5811:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:0.0:1787784718.344482:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784718.344487:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784718.345277:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1787784718.345288:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784718.345926:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658061568 00000100:00100000:2.0:1787784718.345948:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658061568 00000100:00100000:2.0:1787784718.346047:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:2.0:1787784718.346072:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d7c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:24645:x1874627658061568:12345-192.168.203.43@tcp:49:md5sum.0 00000100:00100000:2.0:1787784718.346217:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d7c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:24645:x1874627658061568:12345-192.168.203.43@tcp:49:md5sum.0 Request processed in 146us (2744us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784718.346224:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 178 00000100:00100000:1.0:1787784718.350569:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784718.350611:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.350668:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658061952 00000100:00100000:3.0:1787784718.350677:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658061952 00000100:00100000:3.0:1787784718.350731:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 222 00000100:00100000:3.0:1787784718.350740:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:24645:x1874627658061952:12345-192.168.203.43@tcp:101:md5sum.0 00010000:00010000:3.0:1787784718.350757:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784718.350776:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c08 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784718.350800:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 5242880, mtime: 0x6a8f6e0b, atime: 0x6a8f6e0b, ctime: 0x6a8f6e0b, blocks: 0x2807 00010000:00010000:3.0:1787784718.350820:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=509s v=(0 0) 00010000:00010000:3.0:1787784718.350824:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784718.350833:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.350848:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.350883:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784718.350933:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:24645:x1874627658061952:12345-192.168.203.43@tcp:101:md5sum.0 Request processed in 193us (320us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784718.350941:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 222 00000100:00100000:2.0:1787784718.351310:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658061824 00000100:00100000:2.0:1787784718.351315:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658061824 00000100:00100000:2.0:1787784718.351350:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 179 00000100:00100000:2.0:1787784718.351357:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:24647:x1874627658061824:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1787784718.352104:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x16:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1787784718.353083:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967878 00010000:00010000:2.0:1787784718.353528:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.353546:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.353556:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.353565:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.353571:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.353576:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784718.353678:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787784718.353779:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:24647:x1874627658061824:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 2418us (3215us total) trans 4294967879 rc 0/0 00000100:00100000:2.0:1787784718.353790:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 179 00000020:00080000:2.0:1787784718.353795:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1787784718.359516:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784718.359530:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.359542:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784718.359546:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.359554:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.359574:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff989530cb1d00) 00010000:00010000:3.0:1787784718.359577:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784718.359581:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784718.359588:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5565 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784718.359894:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:2.0:1787784718.361349:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784718.361357:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784718.361360:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784718.361410:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658062080 00000100:00100000:0.0:1787784718.361422:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658062080 00002000:00100000:0.0:1787784718.361484:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x11:0x0] (0->32767) req@ffff98940304b480 x1874627658062080/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784718.361520:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304b480 with x1874627658062080 ext(0->32767) 00002000:00010000:0.0:1787784718.361526:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304b480 00000100:00100000:0.0:1787784718.361562:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 80 00000100:00100000:0.0:1787784718.361572:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658062080:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:2.0:1787784718.367022:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658062080:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 5441us (7122us total) trans 0 rc 32768/32768 00000100:00100000:2.0:1787784718.367043:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 80 00002000:00100000:2.0:1787784718.367071:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x11:0x0] (0->32767) req@ffff98940304b480 x1874627658062080/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/440 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784718.367082:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304b480 with x1874627658062080 ext(0->32767) 00002000:00010000:2.0:1787784718.367089:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304b480 00000100:00100000:1.0:1787784718.380680:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784718.380887:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658062208 00000100:00100000:3.0:1787784718.380898:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658062208 00002000:00100000:3.0:1787784718.381021:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (32768->1048575) req@ffff98940304a1c0 x1874627658062208/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784718.381033:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304a1c0 with x1874627658062208 ext(32768->1048575) 00002000:00010000:3.0:1787784718.381050:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304a1c0 00000100:00100000:3.0:1787784718.381132:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 81 00000100:00100000:3.0:1787784718.381144:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658062208:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:2.0:1787784718.424222:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658062208:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 43071us (43543us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1787784718.424237:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 81 00002000:00100000:2.0:1787784718.424262:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (32768->1048575) req@ffff98940304a1c0 x1874627658062208/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/440 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784718.424273:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304a1c0 with x1874627658062208 ext(32768->1048575) 00002000:00010000:2.0:1787784718.424280:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304a1c0 00000100:00100000:3.0:1787784718.471474:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784718.472626:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658062336 00000100:00100000:2.0:1787784718.472638:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658062336 00002000:00100000:2.0:1787784718.472704:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x11:0x0] (1048576->2097151) req@ffff989402187480 x1874627658062336/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784718.472716:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402187480 with x1874627658062336 ext(1048576->2097151) 00002000:00010000:2.0:1787784718.472722:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402187480 00000100:00100000:2.0:1787784718.472758:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 82 00000100:00100000:2.0:1787784718.472768:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658062336:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:1.0:1787784718.491860:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784718.491912:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784718.491954:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784718.492066:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658062464 00000100:00100000:0.0:1787784718.492076:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658062464 00002000:00100000:0.0:1787784718.492128:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (2097152->3145727) req@ffff989403049680 x1874627658062464/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784718.492139:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403049680 with x1874627658062464 ext(2097152->3145727) 00002000:00010000:0.0:1787784718.492184:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989403049680 00000100:00100000:0.0:1787784718.492213:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658062720 00000100:00100000:0.0:1787784718.492217:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658062720 00002000:00100000:0.0:1787784718.492231:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (4194304->5242879) req@ffff98940304bc00 x1874627658062720/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784718.492239:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304bc00 with x1874627658062720 ext(4194304->5242879) 00002000:00010000:0.0:1787784718.492242:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304bc00 00000100:00100000:0.0:1787784718.492262:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658062592 00000100:00100000:0.0:1787784718.492265:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658062592 00002000:00100000:0.0:1787784718.492278:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (3145728->4194303) req@ffff98940304b840 x1874627658062592/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/0 e 0 to 0 dl 1787784729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784718.492286:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304b840 with x1874627658062592 ext(3145728->4194303) 00002000:00010000:0.0:1787784718.492289:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304b840 00000100:00100000:0.0:1787784718.492301:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 83 00000100:00100000:0.0:1787784718.492310:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658062464:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:3.0:1787784718.493562:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 84 00000100:00100000:3.0:1787784718.493584:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658062720:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:1.0:1787784718.522921:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658062336:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 50144us (51446us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1787784718.522935:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 82 00002000:00100000:1.0:1787784718.522947:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x11:0x0] (1048576->2097151) req@ffff989402187480 x1874627658062336/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/440 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784718.522957:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402187480 with x1874627658062336 ext(1048576->2097151) 00002000:00010000:1.0:1787784718.522963:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402187480 00000100:00100000:1.0:1787784718.522992:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 85 00000100:00100000:1.0:1787784718.523003:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658062592:12345-192.168.203.43@tcp:3:md5sum.0 00000100:00100000:2.0:1787784718.536004:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658062464:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 43631us (44092us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1787784718.536019:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 83 00002000:00100000:2.0:1787784718.536031:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x11:0x0] (2097152->3145727) req@ffff989403049680 x1874627658062464/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/440 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784718.536042:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403049680 with x1874627658062464 ext(2097152->3145727) 00002000:00010000:2.0:1787784718.536047:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989403049680 00000100:00100000:0.0:1787784718.555455:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658062720:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 61866us (63531us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1787784718.555469:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 84 00002000:00100000:0.0:1787784718.555482:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x11:0x0] (4194304->5242879) req@ffff98940304bc00 x1874627658062720/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/440 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784718.555502:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304bc00 with x1874627658062720 ext(4194304->5242879) 00002000:00010000:0.0:1787784718.555508:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304bc00 00000100:00100000:1.0:1787784718.568839:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658062592:12345-192.168.203.43@tcp:3:md5sum.0 Request processed in 45829us (76872us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1787784718.568853:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 85 00002000:00100000:1.0:1787784718.568866:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x11:0x0] (3145728->4194303) req@ffff98940304b840 x1874627658062592/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:599/0 lens 488/440 e 0 to 0 dl 1787784729 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784718.568890:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304b840 with x1874627658062592 ext(3145728->4194303) 00002000:00010000:1.0:1787784718.568896:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98940304b840 00000100:00100000:3.0:1787784718.616331:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784718.617456:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658062976 00000100:00100000:2.0:1787784718.617467:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658062976 00000100:00100000:2.0:1787784718.617533:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 180 00000100:00100000:2.0:1787784718.617544:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:24645:x1874627658062976:12345-192.168.203.43@tcp:35:md5sum.0 00000100:00100000:2.0:1787784718.618026:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:24645:x1874627658062976:12345-192.168.203.43@tcp:35:md5sum.0 Request processed in 482us (1704us total) trans 4294967880 rc 0/0 00000100:00100000:2.0:1787784718.618038:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 180 00010000:00010000:3.0:1787784719.281385:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784719.442585:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967342 is committed 00000001:00080000:3.0:1787784719.442603:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967346 is committed 00000001:00080000:3.0:1787784719.442618:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967349 is committed 00010000:00010000:3.0:1787784720.306171:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784720.882660:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668379520:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784720.882738:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784720.882828:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668379520 00000100:00100000:0.0:1787784720.882837:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668379520 00000100:00100000:0.0:1787784720.882892:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:0.0:1787784720.882902:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668379520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784720.882938:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 73203712 pending 0 free 3852468224 avail 3850371072 00000100:00100000:0.0:1787784720.882996:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668379520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784720.883004:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00080000:0.0:1787784720.883029:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784720.883077:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668379520:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784721.160659:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784721.160708:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658063232 00000100:00100000:0.0:1787784721.160709:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784721.160735:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658063232 00000100:00100000:0.0:1787784721.160744:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784721.160780:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784721.160812:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784721.160830:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:2.0:1787784721.160844:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:2357:x1874627658063232:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784721.161185:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658063488 00000100:00100000:3.0:1787784721.161191:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658063488 00000100:00100000:3.0:1787784721.161231:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658063744 00000100:00100000:3.0:1787784721.161235:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658063744 00000100:00100000:3.0:1787784721.161257:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658063360 00000100:00100000:3.0:1787784721.161260:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658063360 00000100:00100000:3.0:1787784721.161279:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 277 00000100:00100000:3.0:1787784721.161287:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658063488:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784721.161366:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:2357:x1874627658063232:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 524us (717us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784721.161374:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:0.0:1787784721.162715:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784721.162771:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784721.162804:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658063872 00000100:00100000:0.0:1787784721.162812:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658063872 00000100:00100000:0.0:1787784721.162914:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 278 00000100:00100000:0.0:1787784721.162925:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658063744:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784721.163000:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658063744:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 76us (2256us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784721.163007:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 278 00000100:00100000:0.0:1787784721.163025:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 279 00000100:00100000:0.0:1787784721.163032:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658063360:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784721.163069:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658063360:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (2256us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784721.163074:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 279 00000100:00100000:0.0:1787784721.163087:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00100000:0.0:1787784721.163093:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658063872:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784721.163116:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658063872:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784721.163121:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00080000:0.0:1787784721.163142:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784721.163171:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784721.163174:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784721.163185:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658063104 00000100:00100000:0.0:1787784721.163188:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658063104 00000100:00100000:0.0:1787784721.163215:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 535 00000100:00100000:0.0:1787784721.163222:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658063104:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784721.163259:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658063104:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (2478us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784721.163266:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 535 00000100:00100000:2.0:1787784721.163430:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658063616 00000100:00100000:2.0:1787784721.163443:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658063616 00000100:00100000:2.0:1787784721.163512:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:2.0:1787784721.163523:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2357:x1874627658063616:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784721.163896:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2357:x1874627658063616:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 370us (1184us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784721.163904:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00080000:2.0:1787784721.163947:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784721.184993:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658063488:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23686us (24265us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784721.185014:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 277 00010000:00010000:2.0:1787784721.330259:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784722.353919:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784722.419359:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784722.419370:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784722.419960:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 22 count 3 00000040:00080000:0.0:1787784722.419973:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784722.929553:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668379776:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.929646:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784722.929705:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668380288:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.929732:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784722.929767:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1787784722.929774:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668379648:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.929804:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784722.929827:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668380160:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.929850:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784722.929880:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668379904:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.929906:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784722.929925:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784722.929930:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668380032:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.929952:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784722.929987:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668380288 00000100:00100000:0.0:1787784722.929996:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668380288 00000100:00100000:0.0:1787784722.930074:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668380032 00000100:00100000:0.0:1787784722.930078:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668380032 00000100:00100000:0.0:1787784722.930101:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:0.0:1787784722.930128:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668380288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.930218:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668380288:12345-0@lo:400:kworker.0 Request processed in 91us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784722.930227:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:0.0:1787784722.930253:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:0.0:1787784722.930260:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668380032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.930291:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668380032:12345-0@lo:400:kworker.0 Request processed in 31us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784722.930297:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:0.0:1787784722.930318:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668379648 00000100:00100000:0.0:1787784722.930321:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668379648 00000100:00100000:0.0:1787784722.930347:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:0.0:1787784722.930353:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668379648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.930383:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668379648:12345-0@lo:400:kworker.0 Request processed in 29us (577us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784722.930389:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:0.0:1787784722.930446:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668380032:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.930474:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668379648:0@lo:400:kworker.0 00000100:00100000:0.0:1787784722.930483:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967349 gen 2 00000004:00080000:0.0:1787784722.930494:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967347 committed transno 4294967349 00000004:00080000:0.0:1787784722.930510:0:3279:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402b13840, transno 4294967348 00000004:00080000:0.0:1787784722.930521:0:3279:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402b121c0, transno 4294967349 00000100:00100000:0.0:1787784722.930526:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668380288:0@lo:400:kworker.0 00000100:00100000:2.0:1787784722.931351:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668379776 00000100:00100000:2.0:1787784722.931359:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668379776 00000100:00100000:2.0:1787784722.931397:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668380160 00000100:00100000:2.0:1787784722.931401:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668380160 00000100:00100000:2.0:1787784722.931417:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668379904 00000100:00100000:2.0:1787784722.931420:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668379904 00000100:00100000:2.0:1787784722.931439:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:2.0:1787784722.931449:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668379776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784722.931517:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668379776:12345-0@lo:400:kworker.0 Request processed in 69us (1871us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784722.931525:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:2.0:1787784722.931540:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1787784722.931546:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668380160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784722.931575:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668380160:12345-0@lo:400:kworker.0 Request processed in 30us (1724us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784722.931592:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1787784722.931605:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:2.0:1787784722.931611:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668379904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784722.931642:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668379904:12345-0@lo:400:kworker.0 Request processed in 32us (1735us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784722.931648:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00080000:2.0:1787784722.931674:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784722.931679:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784722.931682:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784722.931705:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784722.931719:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668380416:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784722.931757:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784722.931793:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668380416 00000100:00100000:2.0:1787784722.931797:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668380416 00000100:00100000:2.0:1787784722.931824:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1787784722.931830:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668380416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784722.931860:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 73072640 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784722.931897:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668380416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 67us (140us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784722.931903:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1787784722.931943:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668380416:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784722.931974:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668379904:0@lo:400:kworker.0 00000100:00100000:2.0:1787784722.932002:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668380160:0@lo:400:kworker.0 00000100:00100000:2.0:1787784722.932016:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668379776:0@lo:400:kworker.0 00000100:00100000:0.0:1787784723.277806:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784723.278061:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658064000 00000100:00100000:3.0:1787784723.278076:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658064000 00000100:00100000:3.0:1787784723.278176:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 872 00000100:00100000:3.0:1787784723.278188:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25065:x1874627658064000:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784723.278232:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784723.278261:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc556c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c0f expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.278343:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5573 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.278360:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5573 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.278374:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5573 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784723.278688:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 7680 00000004:00200000:3.0:1787784723.278692:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784723.278765:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5573 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.278797:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc556c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c0f expref: 85 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.278815:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5573 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c0f expref: 84 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.278824:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784723.278826:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784723.278830:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784723.279049:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25065:x1874627658064000:12345-192.168.203.43@tcp:101:rm.0 Request processed in 862us (1255us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784723.279058:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 872 00000100:00080000:3.0:1787784723.279106:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784723.290306:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784723.290378:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658064256 00000100:00100000:0.0:1787784723.290392:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658064256 00000100:00100000:0.0:1787784723.290462:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 873 00000100:00100000:0.0:1787784723.290473:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25065:x1874627658064256:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784723.290502:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784723.290529:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc557a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c16 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784723.290860:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 7680 00010000:00010000:0.0:1787784723.292323:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc557a lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c16 expref: 86 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.292350:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:0.0:1787784723.292353:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784723.292359:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc557a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c16 expref: 86 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.292377:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166ec00, rc 0) 00000100:00100000:0.0:1787784723.292467:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:25065:x1874627658064256:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1995us (2177us total) trans 4294967881 rc 301/301 00000100:00100000:0.0:1787784723.292478:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 873 00000100:00100000:0.0:1787784723.297233:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784723.297291:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658064384 00000100:00100000:0.0:1787784723.297304:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658064384 00000100:00100000:0.0:1787784723.297374:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:0.0:1787784723.297385:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:25065:x1874627658064384:12345-192.168.203.43@tcp:37:rm.0 00000100:00100000:0.0:1787784723.313085:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:25065:x1874627658064384:12345-192.168.203.43@tcp:37:rm.0 Request processed in 15301us (15467us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784723.313118:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 181 00000100:00100000:1.0:1787784723.317132:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784723.317349:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658064640 00000100:00100000:0.0:1787784723.317363:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658064640 00000100:00100000:0.0:1787784723.317468:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:0.0:1787784723.317482:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:25065:x1874627658064640:12345-192.168.203.43@tcp:35:rm.0 00000100:00100000:0.0:1787784723.318175:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:25065:x1874627658064640:12345-192.168.203.43@tcp:35:rm.0 Request processed in 666us (1027us total) trans 4294967882 rc 0/0 00000100:00100000:0.0:1787784723.318186:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 182 00000100:00100000:0.0:1787784723.321908:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784723.321987:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658064896 00000100:00100000:3.0:1787784723.322000:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658064896 00000100:00100000:3.0:1787784723.322091:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 874 00000100:00100000:3.0:1787784723.322103:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:25065:x1874627658064896:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784723.322141:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784723.322214:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5588 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c1d expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.322562:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc558f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.322582:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc558f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.322597:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc558f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784723.322684:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 7680 00010000:00010000:3.0:1787784723.322913:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5588 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c1d expref: 89 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.322937:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc558f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c1d expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.322948:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784723.322952:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787784723.322956:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784723.323056:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:25065:x1874627658064896:12345-192.168.203.43@tcp:101:rm.0 Request processed in 955us (1161us total) trans 4294967883 rc 0/0 00000100:00100000:3.0:1787784723.323078:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 874 00000100:00100000:3.0:1787784723.325816:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784723.325961:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658065024 00000100:00100000:0.0:1787784723.325975:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658065024 00000100:00100000:0.0:1787784723.326071:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 875 00000100:00100000:0.0:1787784723.326083:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:25065:x1874627658065024:12345-192.168.203.43@tcp:41:rm.0 00000004:00000024:0.0:1787784723.326365:0:5813:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 7710179328 avail 7697596416 00000100:00100000:0.0:1787784723.326503:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:25065:x1874627658065024:12345-192.168.203.43@tcp:41:rm.0 Request processed in 419us (699us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784723.326512:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 875 00000100:00100000:0.0:1787784723.333448:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784723.335281:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658065152 00000100:00100000:2.0:1787784723.335300:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658065152 00000100:00100000:2.0:1787784723.335458:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 876 00000100:00100000:2.0:1787784723.335472:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:25065:x1874627658065152:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784723.335501:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784723.335530:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc559d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c24 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.335616:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.335633:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.335647:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.335665:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.335673:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.335680:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336194:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc55b2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336206:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc55b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336226:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc55b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784723.336269:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00000004:00200000:2.0:1787784723.336275:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784723.336299:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc55b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336313:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc55b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336321:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336328:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336333:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336340:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784723.336345:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336353:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336366:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784723.336370:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784723.336374:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336389:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336404:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336409:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336415:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336419:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784723.336422:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336427:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336434:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff989530cb1400) 00010000:00010000:2.0:1787784723.336437:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784723.336440:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 2/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336444:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc55ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336484:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fde00/0xb0ce63137acc559d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c24 expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336511:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc55b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c24 expref: 89 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.336519:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:2.0:1787784723.336522:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784723.336524:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fdc00, rc 0) 00000100:00100000:2.0:1787784723.336616:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:25065:x1874627658065152:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1147us (3182us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784723.336626:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 876 00000100:00100000:3.0:1787784723.341836:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784723.344115:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658065408 00000100:00100000:1.0:1787784723.344139:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658065408 00000100:00100000:1.0:1787784723.344267:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 877 00000100:00100000:1.0:1787784723.344281:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:25065:x1874627658065408:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:1.0:1787784723.344365:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787784723.344404:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc55b2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c24 expref: 89 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344438:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc544d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bb4 expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344453:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff989531111e00) 00010000:00010000:1.0:1787784723.344475:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5573 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c0f expref: 87 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344485:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:1.0:1787784723.344488:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784723.344516:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344533:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344568:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344585:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344593:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.344599:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.345041:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.345052:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.345059:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787784723.346092:0:5811:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].8, 64 off8728 00000004:00080000:1.0:1787784723.346109:0:5811:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 514432960445ns, next at 515432956992ns, rc = 0 00010000:00010000:1.0:1787784723.346240:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346255:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346266:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346291:0:5811:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346298:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346304:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346324:0:5811:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787784723.346352:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346358:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346364:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346370:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784723.346375:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346383:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346395:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff98951c407e00) 00010000:00010000:1.0:1787784723.346398:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784723.346403:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346419:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc55c7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346441:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346447:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346453:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346458:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784723.346461:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346467:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346476:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:1.0:1787784723.346478:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784723.346481:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.346487:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc55b9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784723.346497:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000887d6a05 reply state 00000000b8d3d470 transno 4294967884 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784723.346507:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784723.346646:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:25065:x1874627658065408:12345-192.168.203.43@tcp:36:rm.0 Request processed in 2362us (4820us total) trans 4294967884 rc 0/0 00000100:00100000:1.0:1787784723.346656:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 877 00010000:00010000:3.0:1787784723.346785:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784723.346795:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.346804:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784723.346807:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.346815:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.346825:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff98951c407e00) 00010000:00010000:3.0:1787784723.346828:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784723.346831:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 1/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.346837:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc55ce lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.346857:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784723.346859:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784723.346860:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784723.351703:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.351729:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.351738:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.351744:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784723.351750:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.351759:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.351781:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff98951c407400) 00010000:00010000:2.0:1787784723.351784:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784723.351791:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784723.351813:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc55c0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784723.372405:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784723.372533:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658065664 00000100:00100000:1.0:1787784723.372546:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658065664 00000100:00100000:1.0:1787784723.372621:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 183 00000100:00100000:1.0:1787784723.372634:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:25065:x1874627658065664:12345-192.168.203.43@tcp:35:rm.0 00000100:00100000:1.0:1787784723.372986:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:25065:x1874627658065664:12345-192.168.203.43@tcp:35:rm.0 Request processed in 352us (593us total) trans 4294967885 rc 0/0 00000100:00100000:1.0:1787784723.372995:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 183 00000100:00100000:0.0:1787784723.376433:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784723.376595:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658065920 00000100:00100000:3.0:1787784723.376608:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658065920 00000100:00100000:3.0:1787784723.376680:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 878 00000100:00100000:3.0:1787784723.376693:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:25065:x1874627658065920:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:3.0:1787784723.376790:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787784723.376823:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc558f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c1d expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.376849:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc52d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b36 expref: 87 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.376860:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784723.376869:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc52b7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0b2f expref: 86 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.376877:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:3.0:1787784723.376880:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784723.376906:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.376921:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 44 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.376930:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.376981:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 44 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.376995:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 44 pid: 5811 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784723.377036:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668380544:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784723.377230:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504c5b000 going to sleep for 11 seconds 00010000:00010000:3.0:1787784723.377696:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784723.397314:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784723.397374:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066048 00000100:00100000:1.0:1787784723.397387:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066048 00000100:00100000:0.0:1787784723.397452:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668380544:192.168.203.43@tcp:104: 00000100:00080000:3.0:1787784723.397466:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784723.397470:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 45 pid: 5811 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.397485:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784723.397526:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 273 00000100:00100000:1.0:1787784723.397543:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2356:x1874627658066048:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784723.397707:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784723.397715:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 45 pid: 5811 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.397731:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 45 pid: 5811 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.397771:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc5485 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc2 expref: 45 pid: 5811 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.397812:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:1.0:1787784723.397818:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.397829:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.397838:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.397872:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784723.398190:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398212:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398242:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398254:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398263:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784723.398397:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2356:x1874627658066048:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 859us (1098us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784723.398407:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 273 00010000:00010000:0.0:1787784723.398766:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398779:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398789:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398827:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.398842:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 43 pid: 10850 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784723.398860:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668380672:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784723.398927:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503fa0540 going to sleep for 11 seconds 00000100:00100000:0.0:1787784723.404541:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784723.404713:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066176 00000100:00100000:1.0:1787784723.404723:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066176 00000100:00100000:3.0:1787784723.404742:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668380672:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784723.404778:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.404787:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 44 pid: 10850 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784723.404900:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 274 00000100:00100000:1.0:1787784723.404913:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658066176:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784723.404934:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784723.404943:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 44 pid: 10850 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.404959:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 44 pid: 10850 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.405033:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc54a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0bc9 expref: 44 pid: 10850 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.405053:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:1.0:1787784723.405059:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.405070:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.405079:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.405116:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784723.405171:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.405179:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784723.405220:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2357:x1874627658066176:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 309us (693us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784723.405231:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 274 00000004:00200000:3.0:1787784723.409533:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 7680 00010000:00010000:3.0:1787784723.409565:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409578:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409586:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409592:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166e800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784723.409598:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409872:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409913:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:3.0:1787784723.409917:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784723.409923:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409930:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc55e3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409966:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409973:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409979:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409985:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784723.409988:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.409995:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.410002:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:3.0:1787784723.410004:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784723.410007:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784723.410014:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55d5 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784723.410055:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d02ec3c9 reply state 00000000cd0b4109 transno 4294967886 ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784723.410065:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784723.410205:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:25065:x1874627658065920:12345-192.168.203.43@tcp:36:rm.0 Request processed in 33509us (33780us total) trans 4294967886 rc 0/0 00000100:00100000:3.0:1787784723.410216:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 878 00010000:00010000:1.0:1787784723.422083:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.422105:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.422114:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.422122:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166e000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784723.422128:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.422186:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.422208:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff989503a6de00) 00010000:00010000:1.0:1787784723.422212:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784723.422219:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.422227:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc55dc lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784723.481850:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784723.483954:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066304 00000100:00100000:2.0:1787784723.483973:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066304 00000100:00100000:3.0:1787784723.486799:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 879 00000100:00100000:3.0:1787784723.486823:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:24617:x1874627658066304:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1787784723.486989:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:24617:x1874627658066304:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 174us (5153us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784723.487017:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 879 00000001:00080000:2.0:1787784723.496275:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967340 is committed 00000001:00080000:2.0:1787784723.496301:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967339 is committed 00000001:00080000:2.0:1787784723.499423:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967882 is committed 00000001:00080000:2.0:1787784723.499454:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967871 is committed 00000020:00080000:2.0:1787784723.499462:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 13 batchid updated 00000001:00080000:2.0:1787784723.499484:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967885 is committed 00000004:00080000:2.0:1787784723.499496:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 9 committed 00000001:00080000:2.0:1787784723.499519:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967879 is committed 00000001:00080000:2.0:1787784723.499531:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967886 is committed 00000020:00080000:2.0:1787784723.499544:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98953f097b40 refcount 2 committed 0 result 0 batchid 14 00000020:00080000:2.0:1787784723.499549:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895012e9e00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98953f097000 00000020:00080000:2.0:1787784723.499553:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895012e9200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989404052000 00000020:00080000:2.0:1787784723.499578:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 14 committed batchid 13 00000020:00080000:2.0:1787784723.500328:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 13 00000100:00100000:2.0:1787784723.500341:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784723.500367:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668380800:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787784723.500457:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784723.500822:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668380800 00000100:00100000:0.0:1787784723.500831:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668380800 00000100:00100000:0.0:1787784723.500994:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:0.0:1787784723.501007:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668380800:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1787784723.501076:0:6669:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:17 count 1 00010000:00010000:0.0:1787784723.501113:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.501142:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.501172:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=5s v=(0 0) 00010000:00010000:0.0:1787784723.501178:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784723.501183:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 4/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.501219:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.501235:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 11 pid: 10688 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784723.501252:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f083c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668380928:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784723.501374:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503fa03c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784723.503553:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f083c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668380928:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784723.503593:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784723.503997:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784723.504200:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066432 00000100:00100000:1.0:1787784723.504208:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066432 00000100:00080000:3.0:1787784723.504246:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787784723.504263:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 12 pid: 10688 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784723.504302:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 275 00000100:00100000:1.0:1787784723.504312:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658066432:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784723.504330:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784723.504336:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 12 pid: 10688 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.504348:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 12 pid: 10688 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.504376:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc555e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c08 expref: 12 pid: 10688 timeout: 614 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.504406:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784723.504411:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784723.504417:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.504428:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.504464:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784723.504533:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658066432:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 222us (538us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784723.504541:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 275 00010000:00010000:0.0:1787784723.504783:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.504794:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.504813:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.504820:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784723.504841:0:6669:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787784723.505354:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784723.505363:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.505373:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166ec00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784723.505377:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.505388:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.505402:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 1/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.505424:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f505ba1/0xb0ce63137acc55ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784723.505450:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784723.505451:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784723.505453:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784723.509315:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668380800:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8299us (8845us total) trans 4294967341 rc 0/0 00000100:00100000:3.0:1787784723.509334:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1787784723.509625:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967340 gen 2 00000004:00080000:2.0:1787784723.509633:0:3276:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967340 committed transno 4294967340 00000100:00100000:2.0:1787784723.509678:0:3276:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9894021852c0 x1874627668380800/t4294967341(4294967341) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1787784739 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1787784723.509707:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9894021852c0/2, rc 0, transno 4294967341 00000100:00100000:2.0:1787784723.509712:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668380800:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1787784724.402293:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787784725.097897:0:18679:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 18:52:03 (1787784723) 00000100:00100000:1.0:1787784725.323072:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784725.323198:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066560 00000100:00100000:3.0:1787784725.323216:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066560 00000100:00100000:3.0:1787784725.323330:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 880 00000100:00100000:3.0:1787784725.323345:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:25249:x1874627658066560:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784725.323374:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784725.323405:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55f1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c2b expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.323522:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.323539:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.323554:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784725.323825:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784725.323831:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 22 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784725.323924:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.323957:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc55f1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 36 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.323977:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 35 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.323986:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:3.0:1787784725.323989:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784725.323993:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7400, rc 0) 00000100:00100000:3.0:1787784725.324102:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:25249:x1874627658066560:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 762us (1039us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784725.324110:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 880 00000100:00080000:0.0:1787784725.324262:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784725.331127:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784725.331238:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066688 00000100:00100000:2.0:1787784725.331255:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066688 00000100:00100000:2.0:1787784725.331414:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 881 00000100:00100000:2.0:1787784725.331426:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+79:25249:x1874627658066688:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784725.331460:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784725.331488:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55ff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c32 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.331564:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5606 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.331578:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5606 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.331588:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5606 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784725.331874:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784725.331879:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 22 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784725.331968:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5606 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.331998:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc55ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c32 expref: 81 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.332049:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5606 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c32 expref: 80 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.332058:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784725.332061:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784725.332065:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784725.332193:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:25249:x1874627658066688:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 768us (1076us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784725.332202:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 881 00000100:00080000:0.0:1787784725.336316:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784725.349202:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.349488:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066816 00000100:00100000:1.0:1787784725.349501:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066816 00000100:00100000:1.0:1787784725.349586:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 882 00000100:00100000:1.0:1787784725.349598:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:25250:x1874627658066816:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1787784725.349921:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+35:25250:x1874627658066816:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 323us (732us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784725.349931:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 882 00000100:00100000:1.0:1787784725.363383:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784725.363512:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658066944 00000100:00100000:2.0:1787784725.363527:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658066944 00000100:00100000:2.0:1787784725.363605:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 883 00000100:00100000:2.0:1787784725.363616:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:25251:x1874627658066944:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784725.363643:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784725.363714:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc560d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c39 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.363816:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.363835:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.363848:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.363884:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.363891:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.363897:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364441:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364453:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364460:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364467:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784725.364471:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364479:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364494:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784725.364497:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784725.364503:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364510:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5614 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364534:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364540:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364547:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364553:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784725.364556:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364563:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364572:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff989530cb1400) 00010000:00010000:2.0:1787784725.364575:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784725.364578:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364584:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc561b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364625:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc560d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c39 expref: 81 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364638:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784725.364640:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784725.364650:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc560d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c39 expref: 81 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.364662:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fda00, rc 0) 00000100:00100000:2.0:1787784725.365027:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:25251:x1874627658066944:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1410us (1657us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784725.365037:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 883 00000100:00100000:3.0:1787784725.369448:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784725.369588:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658067200 00000100:00100000:0.0:1787784725.369679:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658067200 00000100:00100000:0.0:1787784725.369750:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 884 00000100:00100000:0.0:1787784725.369763:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:25251:x1874627658067200:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784725.369935:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784725.369971:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5606 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c32 expref: 80 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.369997:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:0.0:1787784725.370078:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784725.370944:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.370968:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 34 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.370979:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.371108:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 34 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.371128:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 34 pid: 5812 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784725.371295:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668381056:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784725.371478:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784725.374954:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668381056:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784725.374997:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784725.380751:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784725.381195:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658067328 00000100:00100000:2.0:1787784725.381209:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658067328 00010000:00010000:2.0:1787784725.381268:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 35 pid: 5812 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784725.381320:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 276 00000100:00100000:2.0:1787784725.381331:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:2356:x1874627658067328:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784725.381351:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784725.381358:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 35 pid: 5812 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.381374:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 35 pid: 5812 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.381403:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc55f8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c2b expref: 35 pid: 5812 timeout: 616 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.381424:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784725.381430:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.381454:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.381464:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.381493:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784725.381571:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:2356:x1874627658067328:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 241us (835us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784725.381580:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 276 00010000:00010000:0.0:1787784725.381712:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.381722:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.381755:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.381777:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.381785:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784725.383242:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 512 00010000:00010000:0.0:1787784725.383263:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.383272:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.383280:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.383287:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166e800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784725.383292:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.383301:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.383317:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989505542300) 00010000:00010000:0.0:1787784725.383321:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.383326:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.383333:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5622 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784725.383372:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009f2a0031 reply state 00000000c25f6e34 transno 4294967887 ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784725.383382:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784725.383491:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:25251:x1874627658067200:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 13726us (14051us total) trans 4294967887 rc 0/0 00000100:00100000:0.0:1787784725.383501:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 884 00010000:00010000:0.0:1787784725.395134:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.395193:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.395203:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.395210:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166f800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784725.395216:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.395225:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.395247:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff989503a6d800) 00010000:00010000:0.0:1787784725.395251:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.395256:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.395264:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5629 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784725.400635:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.400755:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658067456 00000100:00100000:1.0:1787784725.400768:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658067456 00000100:00100000:1.0:1787784725.400847:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 885 00000100:00100000:1.0:1787784725.400858:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+80:25252:x1874627658067456:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784725.400891:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784725.400919:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5630 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c40 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.401004:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5637 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.401051:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5637 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.401063:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5637 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784725.401142:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784725.401197:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 23 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784725.401227:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5637 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.401255:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5630 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c40 expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.401275:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5637 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c40 expref: 81 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.401284:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:1.0:1787784725.401287:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784725.401290:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784725.401367:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:25252:x1874627658067456:12345-192.168.203.43@tcp:101:bash.0 Request processed in 511us (748us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784725.401375:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 885 00000100:00100000:2.0:1787784725.405664:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.405832:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658067584 00000100:00100000:1.0:1787784725.405844:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658067584 00000100:00100000:1.0:1787784725.405910:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 886 00000100:00100000:1.0:1787784725.405921:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+81:25252:x1874627658067584:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784725.405950:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784725.405997:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc563e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c47 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406099:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406116:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406130:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406360:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406370:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406377:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406527:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ca00/0xb0ce63137acc5653 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406537:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ca00/0xb0ce63137acc5653 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406544:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ca00/0xb0ce63137acc5653 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784725.406585:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 512 00000004:00200000:1.0:1787784725.406590:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784725.406619:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ca00/0xb0ce63137acc5653 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406644:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406651:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406659:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406666:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.406814:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406824:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.406840:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:1.0:1787784725.406843:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.406848:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407639:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5645 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407676:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407692:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407700:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407724:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1ce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.407728:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407736:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407753:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff98951c407200) 00010000:00010000:1.0:1787784725.407756:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.407760:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407766:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc564c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407827:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc563e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c47 expref: 83 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407844:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ca00/0xb0ce63137acc5653 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c47 expref: 82 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.407851:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:1.0:1787784725.407854:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784725.407856:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1ca00, rc 0) 00000100:00100000:1.0:1787784725.407954:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25252:x1874627658067584:12345-192.168.203.43@tcp:101:bash.0 Request processed in 2033us (2306us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784725.407963:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 886 00000100:00100000:3.0:1787784725.413071:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.413255:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658067712 00000100:00100000:1.0:1787784725.413281:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658067712 00000100:00100000:1.0:1787784725.413350:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 887 00000100:00100000:1.0:1787784725.413360:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25252:x1874627658067712:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784725.413380:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784725.413399:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc565a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c4e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413464:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413478:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413490:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413503:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413510:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413517:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413585:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413592:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413597:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413610:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.413614:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413622:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413634:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:1.0:1787784725.413637:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.413641:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413647:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5661 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413658:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413664:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413669:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413673:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.413676:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413681:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413710:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff98951c407900) 00010000:00010000:1.0:1787784725.413712:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.413715:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413720:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5668 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413749:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc565a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c4e expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413759:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:1.0:1787784725.413761:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.413765:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc565a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c4e expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.413773:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784725.413827:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25252:x1874627658067712:12345-192.168.203.43@tcp:101:bash.0 Request processed in 469us (765us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784725.413835:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 887 00000100:00100000:3.0:1787784725.423223:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784725.423437:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658067840 00000100:00100000:0.0:1787784725.423450:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658067840 00000100:00100000:0.0:1787784725.423525:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 888 00000100:00100000:0.0:1787784725.423538:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+34:25253:x1874627658067840:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784725.423565:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784725.423605:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc566f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c55 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.423692:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.423709:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.423724:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784725.423806:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784725.423810:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 23 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784725.423841:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.423870:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc566f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 36 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.423891:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 35 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.423900:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:0.0:1787784725.423917:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784725.423922:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166e800, rc 0) 00000100:00100000:0.0:1787784725.424489:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+35:25253:x1874627658067840:12345-192.168.203.43@tcp:101:rm.0 Request processed in 468us (795us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784725.424499:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 888 00010000:00010000:0.0:1787784725.426457:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784725.434911:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.435814:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658067968 00000100:00100000:1.0:1787784725.435825:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658067968 00000100:00100000:1.0:1787784725.435889:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 889 00000100:00100000:1.0:1787784725.435898:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+35:25253:x1874627658067968:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784725.435918:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784725.435938:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc567d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c5c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436045:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436060:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436072:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436086:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436093:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436101:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436489:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc5692 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436499:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc5692 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436506:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc5692 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784725.436539:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 512 00000004:00200000:1.0:1787784725.436542:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784725.436563:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc5692 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436585:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436591:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436598:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436604:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.436608:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436616:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436627:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:1.0:1787784725.436630:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.436634:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436640:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5684 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436651:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436657:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436662:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436672:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.436675:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436681:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436688:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff98951c407900) 00010000:00010000:1.0:1787784725.436691:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.436693:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436698:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc568b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.436730:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc567d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c5c expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.437228:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d000/0xb0ce63137acc5692 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c5c expref: 36 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.437237:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:1.0:1787784725.437239:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784725.437241:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d000, rc 0) 00000100:00100000:1.0:1787784725.437309:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25253:x1874627658067968:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1411us (2405us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784725.437317:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 889 00000100:00100000:3.0:1787784725.440560:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784725.443879:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658068096 00000100:00100000:0.0:1787784725.443900:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658068096 00000100:00100000:0.0:1787784725.443992:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 890 00000100:00100000:0.0:1787784725.444088:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25253:x1874627658068096:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784725.444117:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784725.444164:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5699 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c63 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444256:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444275:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444289:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444310:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444399:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444408:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444512:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444521:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444529:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444536:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166ea00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784725.444541:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444550:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444565:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:0.0:1787784725.444568:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.444574:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444581:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444669:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444676:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444682:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444694:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166e000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784725.444697:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444703:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444712:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff989503a6d800) 00010000:00010000:0.0:1787784725.444714:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.444717:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444723:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444765:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5699 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c63 expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444776:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:0.0:1787784725.444779:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.444783:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc5699 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c63 expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.444795:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166f800, rc 0) 00000100:00100000:0.0:1787784725.444963:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25253:x1874627658068096:12345-192.168.203.43@tcp:101:rm.0 Request processed in 960us (4416us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784725.444971:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 890 00000100:00100000:1.0:1787784725.449396:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784725.449609:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658068224 00000100:00100000:3.0:1787784725.449620:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658068224 00000100:00100000:3.0:1787784725.449680:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 891 00000100:00100000:3.0:1787784725.449690:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25253:x1874627658068224:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784725.449710:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784725.449740:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc56ae lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c6a expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449809:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449824:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449835:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449850:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449858:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449865:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449927:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449941:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449947:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449954:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784725.449958:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449966:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.449979:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:3.0:1787784725.449982:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784725.449987:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450120:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc56b5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450135:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450141:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450178:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450184:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784725.450194:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450203:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450214:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff989505542800) 00010000:00010000:3.0:1787784725.450217:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784725.450220:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450227:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc56bc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450262:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc56ae lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c6a expref: 37 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450273:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:3.0:1787784725.450276:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784725.450280:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc56ae lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c6a expref: 37 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.450289:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784725.450551:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25253:x1874627658068224:12345-192.168.203.43@tcp:101:rm.0 Request processed in 860us (1161us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784725.450560:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 891 00000100:00100000:2.0:1787784725.466970:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784725.467292:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658068352 00000100:00100000:0.0:1787784725.467334:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658068352 00000100:00100000:0.0:1787784725.467429:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 892 00000100:00100000:0.0:1787784725.467442:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25254:x1874627658068352:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784725.467470:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784725.467500:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc56c3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c71 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467591:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467618:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467632:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467650:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467658:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467665:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467763:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467771:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467778:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467791:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166e000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784725.467796:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467805:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467819:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:0.0:1787784725.467822:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.467828:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467834:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc56ca lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467851:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467857:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467863:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467868:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166ea00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784725.467871:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467877:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467898:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff989503a6d800) 00010000:00010000:0.0:1787784725.467900:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.467903:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467909:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc56d1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467951:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc56c3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c71 expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467962:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:0.0:1787784725.467964:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784725.467968:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc56c3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c71 expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.467978:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166f800, rc 0) 00000100:00100000:0.0:1787784725.468091:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25254:x1874627658068352:12345-192.168.203.43@tcp:101:cp.0 Request processed in 651us (1129us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784725.468100:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 892 00000100:00100000:2.0:1787784725.470472:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.470548:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658068480 00000100:00100000:1.0:1787784725.470565:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658068480 00000100:00100000:1.0:1787784725.470656:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 893 00000100:00100000:1.0:1787784725.470668:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25254:x1874627658068480:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784725.470751:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784725.470779:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc56d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c78 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.470866:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.470884:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.470899:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.470921:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.470929:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.470937:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471099:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471108:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471138:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471193:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.471200:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471210:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471227:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:1.0:1787784725.471230:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.471236:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471243:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc56df lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471262:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471269:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471275:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471281:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.471284:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471317:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471327:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff98951c407900) 00010000:00010000:1.0:1787784725.471329:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.471332:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471338:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc56e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471383:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc56d8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c78 expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471413:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:1.0:1787784725.471417:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.471421:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc56d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c78 expref: 37 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.471432:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784725.471512:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25254:x1874627658068480:12345-192.168.203.43@tcp:101:cp.0 Request processed in 845us (1048us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784725.471523:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 893 00000100:00100000:1.0:1787784725.479092:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784725.479242:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658068736 00000100:00100000:2.0:1787784725.479256:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658068736 00000100:00100000:2.0:1787784725.479434:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 894 00000100:00100000:2.0:1787784725.479447:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25254:x1874627658068736:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784725.479475:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784725.479502:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc56ed lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c7f expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.479811:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.479831:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.479844:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.479882:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.479890:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.479897:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784725.484350:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x14:0x0], index 1: 0 00010000:00010000:2.0:1787784725.485477:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc5702 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.485502:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc5702 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.485516:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc5702 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.485567:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.485574:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.485581:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.485587:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784725.486359:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.486396:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.486415:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:2.0:1787784725.486419:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784725.486425:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.486433:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc56f4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784725.486455:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000433e1491 reply state 0000000014bf47d2 transno 4294967888 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784725.486467:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784725.486527:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc56ed lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c7f expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.486545:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fda00/0xb0ce63137acc5702 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c7f expref: 38 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.486553:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784725.486556:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784725.486559:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fda00, rc 0) 00000100:00100000:2.0:1787784725.486992:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:25254:x1874627658068736:12345-192.168.203.43@tcp:101:cp.0 Request processed in 7545us (7911us total) trans 4294967888 rc 0/0 00000100:00100000:2.0:1787784725.487003:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 894 00000040:00080000:0.0:1787784725.491454:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784725.491465:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784725.491891:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784725.491903:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787784725.493975:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.493992:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.494030:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.494037:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784725.494042:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.494050:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.494065:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff989530cb1400) 00010000:00010000:1.0:1787784725.494068:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784725.494072:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.494078:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc56fb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784725.495365:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784725.495433:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658068864 00000100:00100000:0.0:1787784725.495449:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658068864 00000100:00100000:0.0:1787784725.495536:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 224 00000100:00080000:3.0:1787784725.495543:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784725.495549:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658068864:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784725.495592:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784725.495619:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c86 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784725.495650:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x14:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784725.495671:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=516s v=(0 0) 00010000:00010000:0.0:1787784725.495675:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784725.495694:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.495709:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.495722:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166f800, rc 0) 00000100:00100000:0.0:1787784725.495793:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658068864:12345-192.168.203.43@tcp:101:cp.0 Request processed in 248us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784725.495800:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 224 00000100:00100000:2.0:1787784725.497912:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784725.498046:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658068992 00000100:00100000:0.0:1787784725.498058:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658068992 00000100:00100000:0.0:1787784725.498125:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 225 00000100:00100000:0.0:1787784725.498136:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:25254:x1874627658068992:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784725.498200:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784725.498227:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c8d expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.498269:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.498297:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784725.498301:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784725.498314:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.498354:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.498371:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 11 pid: 6670 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784725.498388:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668381184:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784725.498495:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784725.500847:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668381184:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784725.500897:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784725.500923:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166ea00, rc 0) 00000100:00100000:0.0:1787784725.500996:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:25254:x1874627658068992:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2860us (3095us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784725.501015:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 225 00000100:00100000:1.0:1787784725.502675:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784725.502805:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658069120 00000100:00100000:3.0:1787784725.502816:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658069120 00010000:00010000:3.0:1787784725.502863:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 11 pid: 6670 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784725.502935:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 277 00000100:00100000:3.0:1787784725.502945:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658069120:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784725.502963:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784725.502993:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 11 pid: 6670 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.503020:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 11 pid: 6670 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.503049:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5710 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c86 expref: 11 pid: 6670 timeout: 616 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.503071:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784725.503076:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784725.503086:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.503098:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.503198:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784725.503216:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668381312:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784725.503315:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72480 going to sleep for 11 seconds 00000100:00100000:3.0:1787784725.505557:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668381312:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784725.505600:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784725.505664:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658069120:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2719us (2999us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784725.506071:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 277 00000100:00100000:0.0:1787784725.507533:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.507648:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658069376 00000100:00100000:1.0:1787784725.507659:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658069376 00000100:00100000:0.0:1787784725.507760:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 184 00000100:00100000:0.0:1787784725.507773:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d5050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25254:x1874627658069376:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:0.0:1787784725.508421:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d5050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:25254:x1874627658069376:12345-192.168.203.43@tcp:35:cp.0 Request processed in 647us (889us total) trans 4294967889 rc 0/0 00000100:00100000:0.0:1787784725.508431:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 184 00000100:00100000:1.0:1787784725.528652:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784725.528867:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658069504 00000100:00100000:3.0:1787784725.528880:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658069504 00000100:00100000:3.0:1787784725.528984:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 895 00000100:00100000:3.0:1787784725.529004:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25257:x1874627658069504:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784725.529043:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784725.529071:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc571e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c94 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529212:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529229:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529243:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529259:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529268:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529274:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529426:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5733 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529436:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5733 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.529444:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5733 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784725.529573:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x17:0x0]: returning size 158 00000004:00200000:2.0:1787784725.529585:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784725.529612:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5733 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529626:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5733 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529635:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529641:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529646:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529652:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784725.529656:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529664:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529733:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:2.0:1787784725.529737:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784725.529742:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529749:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5725 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529832:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529851:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529857:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529861:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784725.529865:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529870:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529878:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff989505542800) 00010000:00010000:2.0:1787784725.529881:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784725.529884:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529890:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc572c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529930:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc571e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c94 expref: 84 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529945:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5733 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c94 expref: 83 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.529953:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784725.529955:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784725.529964:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:2.0:1787784725.530063:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25257:x1874627658069504:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1061us (1425us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784725.530071:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 895 00000100:00100000:2.0:1787784725.533830:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784725.534034:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658069632 00000100:00100000:3.0:1787784725.534047:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658069632 00000100:00100000:3.0:1787784725.534136:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 226 00000100:00100000:3.0:1787784725.534184:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627658069632:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784725.534208:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784725.534232:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc573a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0c9b expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784725.534310:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668381440:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784725.534418:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72480 going to sleep for 11 seconds 00000100:00080000:0.0:1787784725.534666:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:3.0:1787784725.536920:0:6669:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1787784725.536933:0:6669:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb mtime: 0 -> 1787784724 00002000:00010000:3.0:1787784725.536935:0:6669:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb atime: 0 -> 1787784724 00002000:00010000:3.0:1787784725.536938:0:6669:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb ctime: 0 -> 1787784724 00000100:00100000:3.0:1787784725.536967:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668381440:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784725.537023:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc573a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c9b expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.537079:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc573a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c9b expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784725.537096:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784725.537478:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627658069632:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3329us (3660us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784725.537489:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 226 00000100:00100000:3.0:1787784725.539963:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.540084:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658069760 00000100:00100000:1.0:1787784725.540096:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658069760 00000100:00100000:1.0:1787784725.540209:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 896 00000100:00100000:1.0:1787784725.540221:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25257:x1874627658069760:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784725.540254:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784725.540281:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5741 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ca2 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.540378:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc5748 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.540396:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc5748 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.540409:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc5748 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784725.540494:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 512 00000004:00200000:1.0:1787784725.540499:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784725.540528:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc5748 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.540561:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5741 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ca2 expref: 85 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.540583:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1ce00/0xb0ce63137acc5748 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ca2 expref: 84 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784725.540651:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:1.0:1787784725.540654:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784725.540659:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1ce00, rc 0) 00000100:00100000:1.0:1787784725.540740:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25257:x1874627658069760:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 520us (795us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784725.540749:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 896 00000100:00100000:2.0:1787784725.543513:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784725.543632:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658069888 00000100:00100000:0.0:1787784725.543645:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658069888 00000100:00100000:0.0:1787784725.543872:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 227 00000100:00100000:0.0:1787784725.543885:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658069888:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784725.543909:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784725.543932:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc574f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ca9 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784725.544026:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668381568:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784725.544183:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507d67780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784725.547035:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668381568:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784725.547086:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc574f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ca9 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.547117:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc574f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ca9 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.547136:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166f800, rc 0) 00000100:00100000:2.0:1787784725.547434:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658069888:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3549us (3932us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784725.547446:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 227 00000100:00100000:0.0:1787784725.553714:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784725.553788:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658070144 00000100:00100000:2.0:1787784725.553800:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658070144 00000100:00100000:2.0:1787784725.553896:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 35 00000100:00100000:2.0:1787784725.553910:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25257:x1874627658070144:12345-192.168.203.43@tcp:103:lfs.0 00010000:00010000:2.0:1787784725.553932:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784725.553966:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5733 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c94 expref: 84 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784725.553991:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784725.554033:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784725.555288:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25257:x1874627658070144:12345-192.168.203.43@tcp:103:lfs.0 Request processed in 1372us (1580us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784725.555304:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 35 00000100:00100000:0.0:1787784725.557861:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784725.557979:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658070272 00000100:00100000:1.0:1787784725.557993:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658070272 00000100:00100000:1.0:1787784725.558137:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 897 00000100:00100000:1.0:1787784725.558187:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25257:x1874627658070272:12345-192.168.203.43@tcp:58:lfs.0 00000100:00100000:1.0:1787784725.558373:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25257:x1874627658070272:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 226us (527us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784725.558381:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 897 00000100:00100000:3.0:1787784726.005222:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668381696:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784726.005332:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784726.005434:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668381696 00000100:00100000:3.0:1787784726.005443:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668381696 00000100:00100000:3.0:1787784726.005515:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:3.0:1787784726.005527:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668381696:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784726.005574:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 73203712 pending 0 free 3852468224 avail 3850371072 00000100:00100000:3.0:1787784726.007576:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668381696:12345-0@lo:13:osp-pre-0-0.0 Request processed in 2040us (2233us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784726.007724:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00080000:3.0:1787784726.007774:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784726.007843:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668381696:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784726.282430:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.282474:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.282496:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.282528:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.282590:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658070528 00000100:00100000:0.0:1787784726.282603:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658070528 00000100:00100000:0.0:1787784726.282677:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 334 00000100:00100000:0.0:1787784726.282690:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:2355:x1874627658070528:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784726.282792:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:2355:x1874627658070528:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 105us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.282801:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 334 00000100:00100000:0.0:1787784726.282836:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658070784 00000100:00100000:0.0:1787784726.282840:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658070784 00000100:00100000:0.0:1787784726.282866:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658071040 00000100:00100000:0.0:1787784726.282879:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658071040 00000100:00100000:0.0:1787784726.282917:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 283 00000100:00100000:0.0:1787784726.282924:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627658070784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784726.282963:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627658070784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (488us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.282970:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 283 00000100:00100000:0.0:1787784726.282983:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00100000:0.0:1787784726.282990:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658071040:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784726.283102:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658071040:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 111us (604us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.283108:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00080000:0.0:1787784726.283446:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784726.290912:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.291024:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658070656 00000100:00100000:0.0:1787784726.291037:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658070656 00000100:00100000:0.0:1787784726.291112:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:0.0:1787784726.291124:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627658070656:12345-192.168.203.43@tcp:400:kworker.0 00000040:00080000:3.0:1787784726.291753:0:18728:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784726.291760:0:18728:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784726.291812:0:18728:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 22 count 3 00000040:00080000:3.0:1787784726.291821:0:18728:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784726.293623:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658070400 00000100:00100000:2.0:1787784726.293643:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658070400 00000100:00100000:2.0:1787784726.293724:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 537 00000100:00100000:2.0:1787784726.293736:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658070400:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784726.293953:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2355:x1874627658070656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 2820us (3044us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.293964:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00080000:0.0:1787784726.294017:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784726.294542:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.294612:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.294637:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658071168 00000100:00100000:1.0:1787784726.294642:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658071168 00000100:00100000:1.0:1787784726.294681:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:1.0:1787784726.294690:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658071168:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784726.294739:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658071168:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (127us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784726.294747:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:3.0:1787784726.294861:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658070912 00000100:00100000:3.0:1787784726.294865:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658070912 00000100:00100000:3.0:1787784726.294905:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:3.0:1787784726.294913:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2355:x1874627658070912:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784726.294966:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2355:x1874627658070912:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 54us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784726.294974:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:2.0:1787784726.300892:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658070400:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 7140us (18344us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784726.300913:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 537 00000100:00100000:1.0:1787784726.348649:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784726.348741:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658071296 00000100:00100000:3.0:1787784726.348754:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658071296 00000100:00100000:3.0:1787784726.348826:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 898 00000100:00100000:3.0:1787784726.348837:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25267:x1874627658071296:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784726.348867:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784726.348893:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5756 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cb0 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.348983:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349015:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349031:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349047:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349054:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349081:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349248:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc576b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349257:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc576b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349264:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc576b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784726.349305:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x17:0x0]: returning size 158 00000004:00200000:3.0:1787784726.349309:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784726.349331:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc576b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349345:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc576b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349354:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349359:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349366:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349372:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784726.349376:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349391:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349405:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff98951c407d00) 00010000:00010000:3.0:1787784726.349408:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784726.349415:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349421:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc575d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349438:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349444:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349451:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349456:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784726.349459:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349466:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349473:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff989505542d00) 00010000:00010000:3.0:1787784726.349476:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784726.349479:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349495:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5764 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349535:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5756 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cb0 expref: 85 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349550:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc576b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cb0 expref: 84 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.349558:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:3.0:1787784726.349561:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784726.349563:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784726.349646:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25267:x1874627658071296:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 811us (1012us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784726.349654:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 898 00000100:00100000:1.0:1787784726.352780:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784726.352898:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658071424 00000100:00100000:2.0:1787784726.352910:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658071424 00000100:00100000:2.0:1787784726.352989:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 228 00000100:00100000:2.0:1787784726.353018:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658071424:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784726.353043:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784726.353070:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc5772 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cb7 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784726.353185:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668381824:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784726.353300:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784726.355451:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668381824:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784726.355503:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc5772 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cb7 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.355561:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc5772 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cb7 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.355576:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784726.355653:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658071424:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2638us (2888us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784726.355663:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 228 00000100:00100000:1.0:1787784726.359096:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784726.359254:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658071552 00000100:00100000:2.0:1787784726.359266:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658071552 00000100:00100000:2.0:1787784726.359340:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 229 00000100:00100000:2.0:1787784726.359353:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2354:x1874627658071552:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784726.359378:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784726.359403:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5779 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cbe expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784726.359480:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668381952:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784726.359581:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784726.369463:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668381952:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784726.369518:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5779 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cbe expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.369550:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5779 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cbe expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.369929:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:2.0:1787784726.370025:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2354:x1874627658071552:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 10673us (10943us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784726.370037:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 229 00000100:00100000:3.0:1787784726.379929:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784726.380043:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658071808 00000100:00100000:2.0:1787784726.380055:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658071808 00000100:00100000:2.0:1787784726.380137:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 899 00000100:00100000:2.0:1787784726.380176:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25267:x1874627658071808:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:2.0:1787784726.380528:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].22, 136 off11184 00000100:00100000:2.0:1787784726.380631:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25267:x1874627658071808:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 483us (715us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784726.380640:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 899 00010000:00010000:2.0:1787784726.450416:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784726.515388:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784726.515398:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784726.515638:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 4 00000040:00080000:2.0:1787784726.515653:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784726.516055:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539377480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668382080:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784726.516271:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f3b5540 going to sleep for 11 seconds 00000100:00100000:1.0:1787784726.531271:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539377480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668382080:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784726.531605:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.531902:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658071936 00000100:00100000:1.0:1787784726.531911:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658071936 00000100:00100000:1.0:1787784726.532045:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 900 00000100:00100000:1.0:1787784726.532068:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:25277:x1874627658071936:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784726.532099:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x17:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784726.532231:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:25277:x1874627658071936:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 163us (633us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784726.532239:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 900 00000100:00100000:2.0:1787784726.544105:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.546428:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658072192 00000100:00100000:0.0:1787784726.546444:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658072192 00000100:00100000:0.0:1787784726.546519:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 901 00000100:00100000:0.0:1787784726.546531:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:25277:x1874627658072192:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784726.546554:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784726.546598:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc5780 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cc5 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.546704:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc5787 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.546719:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc5787 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.546732:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc5787 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784726.546743:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x17:0x0]: returning size 158 00010000:00010000:0.0:1787784726.546874:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc5780 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cc5 expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.546901:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc5787 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cc5 expref: 41 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.546910:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:0.0:1787784726.546913:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784726.546916:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166e000, rc 0) 00000100:00100000:0.0:1787784726.546991:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25277:x1874627658072192:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 464us (2901us total) trans 4294967890 rc 0/0 00000100:00100000:0.0:1787784726.547012:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 901 00000100:00100000:1.0:1787784726.551539:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.552242:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658072320 00000100:00100000:0.0:1787784726.552254:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658072320 00000100:00100000:0.0:1787784726.552310:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:0.0:1787784726.552321:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658072320:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784726.552403:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.552423:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.552432:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784726.552437:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784726.552442:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.552478:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.552492:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 10 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784726.552506:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668382208:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784726.552590:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950310d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787784726.572573:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784726.572641:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668382208:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784726.572691:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784726.583404:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658072448 00000100:00100000:2.0:1787784726.583425:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658072448 00002000:00100000:2.0:1787784726.583494:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x14:0x0] (0->157) req@ffff989401835a40 x1874627658072448/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:607/0 lens 648/0 e 0 to 0 dl 1787784737 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784726.583520:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401835a40 with x1874627658072448 ext(0->157) 00010000:00010000:2.0:1787784726.583532:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 11 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.583552:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 11 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784726.583562:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989401835a40 00000100:00100000:2.0:1787784726.583592:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 7 00000100:00100000:2.0:1787784726.583602:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658072448:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784726.584760:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658072448:12345-192.168.203.43@tcp:4:cp.0 Request processed in 1148us (12186us total) trans 4294967350 rc 0/0 00000100:00100000:2.0:1787784726.584775:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 7 00002000:00100000:2.0:1787784726.584787:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x14:0x0] (0->157) req@ffff989401835a40 x1874627658072448/t4294967350(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:607/0 lens 648/448 e 0 to 0 dl 1787784737 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784726.584799:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401835a40 with x1874627658072448 ext(0->157) 00010000:00010000:2.0:1787784726.584808:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 13 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.584827:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 13 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784726.584835:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989401835a40 00000100:00100000:0.0:1787784726.589474:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784726.590600:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658072576 00000100:00100000:2.0:1787784726.590625:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658072576 00000100:00100000:2.0:1787784726.590710:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 36 00000100:00100000:2.0:1787784726.590721:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ef850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2357:x1874627658072576:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784726.590740:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784726.590774:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc5787 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cc5 expref: 41 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.590796:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:2.0:1787784726.590801:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784726.590878:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ef850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2357:x1874627658072576:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 158us (1413us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784726.590885:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 36 00000100:00100000:0.0:1787784726.598365:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784726.598591:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658072704 00000100:00100000:2.0:1787784726.598608:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658072704 00010000:00010000:2.0:1787784726.598664:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 13 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784726.598717:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 278 00000100:00100000:2.0:1787784726.598728:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658072704:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784726.598747:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784726.598774:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 13 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.598789:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 13 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.598818:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ab40707e/0xb0ce63137acc5717 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c8d expref: 13 pid: 10688 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.598853:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784726.598858:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784726.598865:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.598876:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784726.598911:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784726.598949:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.598967:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784726.598993:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658072704:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 263us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784726.599021:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 278 00010000:00010000:0.0:1787784726.599053:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.599064:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.599073:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.599089:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166f800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784726.599094:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.599102:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.599123:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.599131:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093917ff7/0xb0ce63137acc5795 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784726.599236:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658072320:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 46914us (47705us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.599255:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:0.0:1787784726.611079:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.611134:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658072960 00000100:00100000:0.0:1787784726.611176:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658072960 00000100:00100000:0.0:1787784726.611254:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 902 00000100:00100000:0.0:1787784726.611272:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:25277:x1874627658072960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784726.611420:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:25277:x1874627658072960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 155us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.611430:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 902 00000100:00100000:1.0:1787784726.620574:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.620726:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658073088 00000100:00100000:0.0:1787784726.620737:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658073088 00000100:00100000:0.0:1787784726.620803:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 231 00000100:00100000:0.0:1787784726.620815:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658073088:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784726.620832:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784726.620854:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000736d4f54/0xb0ce63137acc579c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ccc expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784726.620881:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x14:0x0] initial LVB size: 158, mtime: 0x6a8f6e14, atime: 0x0, ctime: 0x6a8f6e14, blocks: 0x1 00010000:00010000:0.0:1787784726.620901:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=517s v=(0 0) 00010000:00010000:0.0:1787784726.620905:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784726.620916:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000736d4f54/0xb0ce63137acc579c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ccc expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.620933:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000736d4f54/0xb0ce63137acc579c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ccc expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.620965:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031a00, rc 0) 00000100:00100000:0.0:1787784726.621039:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658073088:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 227us (475us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.621047:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 231 00000100:00100000:1.0:1787784726.629564:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.629617:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658073344 00000100:00100000:1.0:1787784726.629629:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658073344 00000100:00100000:1.0:1787784726.629724:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 903 00000100:00100000:1.0:1787784726.629738:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:25277:x1874627658073344:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784726.629877:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:25277:x1874627658073344:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 143us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784726.629896:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 903 00000100:00100000:0.0:1787784726.636578:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.636660:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658073472 00000100:00100000:1.0:1787784726.636679:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658073472 00002000:00100000:1.0:1787784726.636750:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x14:0x0] (0->4095) req@ffff989518f09680 x1874627658073472/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:607/0 lens 488/0 e 0 to 0 dl 1787784737 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784726.636765:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f09680 with x1874627658073472 ext(0->4095) 00002000:00010000:1.0:1787784726.636771:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989518f09680 00000100:00100000:1.0:1787784726.636815:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 86 00000100:00100000:1.0:1787784726.636825:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658073472:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784726.637865:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658073472:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 1032us (1288us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784726.637880:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 86 00002000:00100000:3.0:1787784726.637891:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x14:0x0] (0->4095) req@ffff989518f09680 x1874627658073472/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:607/0 lens 488/4536 e 0 to 0 dl 1787784737 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784726.637916:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f09680 with x1874627658073472 ext(0->4095) 00002000:00010000:3.0:1787784726.637922:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989518f09680 00000100:00100000:1.0:1787784726.647108:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.647331:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658073600 00000100:00100000:0.0:1787784726.647344:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658073600 00000100:00100000:0.0:1787784726.647425:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 904 00000100:00100000:0.0:1787784726.647441:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:25277:x1874627658073600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784726.647470:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784726.647499:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57a3 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cd3 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.647545:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.647560:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.647573:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.648044:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57a3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 42 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.648076:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.648094:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:0.0:1787784726.648098:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784726.648103:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031800, rc 0) 00000100:00100000:0.0:1787784726.648259:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25277:x1874627658073600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 823us (1166us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.648269:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 904 00000100:00100000:0.0:1787784726.654978:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784726.655274:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658073728 00000100:00100000:1.0:1787784726.655292:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658073728 00000100:00100000:1.0:1787784726.655381:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 905 00000100:00100000:1.0:1787784726.655393:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25277:x1874627658073728:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784726.655440:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x17:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784726.655609:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25277:x1874627658073728:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 217us (639us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784726.655630:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 905 00000100:00100000:1.0:1787784726.676334:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784726.678128:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658073856 00000100:00100000:3.0:1787784726.678142:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658073856 00000100:00100000:3.0:1787784726.678226:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 232 00000100:00100000:3.0:1787784726.678237:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658073856:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784726.678280:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678298:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784726.678303:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784726.678308:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678322:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678362:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678371:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678380:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678404:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784726.678408:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678597:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678615:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784726.678622:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc57b1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784726.678766:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658073856:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 530us (2440us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784726.678774:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 232 00000100:00100000:3.0:1787784726.684420:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.684505:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658074112 00000100:00100000:0.0:1787784726.684515:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658074112 00000100:00100000:0.0:1787784726.684566:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 906 00000100:00100000:0.0:1787784726.684576:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25277:x1874627658074112:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784726.684656:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.684675:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.684684:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.684718:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.684732:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 41 pid: 10850 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784726.684766:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668382336:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784726.684855:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950310d840 going to sleep for 11 seconds 00000100:00100000:1.0:1787784726.691671:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.691803:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668382336:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784726.691830:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784726.694859:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658074240 00000100:00100000:3.0:1787784726.694874:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658074240 00010000:00010000:0.0:1787784726.701284:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 42 pid: 10850 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784726.701348:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 279 00000100:00100000:0.0:1787784726.701359:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:2357:x1874627658074240:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784726.701374:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784726.701381:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 42 pid: 10850 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.701395:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 42 pid: 10850 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.701420:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc57aa lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cd3 expref: 42 pid: 10850 timeout: 617 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.701438:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:0.0:1787784726.701444:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.701474:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.701481:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784726.701509:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784726.702231:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2357:x1874627658074240:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 871us (10565us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784726.702241:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 279 00010000:00010000:1.0:1787784726.702324:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702334:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702521:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702529:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702535:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702664:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784726.702670:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702678:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702701:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:1.0:1787784726.702704:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784726.702708:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784726.702714:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc57b8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784726.702912:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25277:x1874627658074112:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 18336us (18501us total) trans 4294967891 rc 0/0 00000100:00100000:1.0:1787784726.702921:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 906 00000040:00080000:0.0:1787784726.703362:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784726.703367:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784726.704358:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1787784726.704373:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784726.709813:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784726.709961:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658074496 00000100:00100000:0.0:1787784726.709971:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658074496 00000100:00100000:0.0:1787784726.710041:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 185 00000100:00100000:0.0:1787784726.710052:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25277:x1874627658074496:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784726.710315:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:25277:x1874627658074496:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 264us (509us total) trans 4294967892 rc 0/0 00000100:00100000:0.0:1787784726.710324:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 185 00000040:00080000:3.0:1787784727.131045:0:18778:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784727.131056:0:18778:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784727.131104:0:18778:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 4 00000040:00080000:3.0:1787784727.131110:0:18778:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784727.214657:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784727.214731:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658074624 00000100:00100000:2.0:1787784727.214741:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658074624 00000100:00100000:2.0:1787784727.214799:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 233 00000100:00100000:2.0:1787784727.214810:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658074624:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784727.214826:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784727.214843:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009199ca5b/0xb0ce63137acc57bf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cda expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784727.214868:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784727.214872:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784727.214880:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009199ca5b/0xb0ce63137acc57bf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cda expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784727.214912:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009199ca5b/0xb0ce63137acc57bf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cda expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784727.214927:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd400, rc 0) 00000100:00100000:2.0:1787784727.214974:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658074624:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 166us (327us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784727.214981:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 233 00000100:00080000:1.0:1787784727.215047:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784727.220704:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784727.220818:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658074880 00000100:00100000:0.0:1787784727.220828:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658074880 00000100:00100000:0.0:1787784727.220911:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 37 00000100:00100000:0.0:1787784727.220923:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25278:x1874627658074880:12345-192.168.203.43@tcp:103:lfs.0 00010000:00010000:0.0:1787784727.220937:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784727.220969:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc576b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cb0 expref: 84 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784727.220989:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff989530cb1000) 00010000:00010000:0.0:1787784727.221008:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784727.221058:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25278:x1874627658074880:12345-192.168.203.43@tcp:103:lfs.0 Request processed in 138us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784727.221067:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 37 00000100:00080000:3.0:1787784727.221278:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784727.226836:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784727.226915:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075008 00000100:00100000:2.0:1787784727.226924:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075008 00000100:00100000:2.0:1787784727.226984:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 907 00000100:00100000:2.0:1787784727.227008:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25278:x1874627658075008:12345-192.168.203.43@tcp:58:lfs.0 00000100:00100000:2.0:1787784727.227126:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25278:x1874627658075008:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 133us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784727.227134:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 907 00010000:00010000:3.0:1787784727.474162:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784727.729583:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784727.729592:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784727.730207:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1787784727.730219:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:2.0:1787784727.915826:0:18829:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784727.915836:0:18829:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784727.915966:0:18829:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 4 00000040:00080000:2.0:1787784727.916278:0:18829:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784728.051910:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668382464:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784728.051985:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784728.052221:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668382464 00000100:00100000:0.0:1787784728.052230:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668382464 00000100:00100000:0.0:1787784728.052294:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:0.0:1787784728.052304:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668382464:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784728.052343:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 76480512 pending 1703936 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784728.052390:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668382464:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (406us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784728.052397:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00080000:3.0:1787784728.052464:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784728.052503:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668382464:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784728.053333:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784728.053384:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784728.053397:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668382720:0@lo:400:kworker.0 00000100:00100000:2.0:1787784728.053442:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784728.053486:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668382720 00000100:00100000:2.0:1787784728.053491:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668382720 00000100:00100000:2.0:1787784728.053539:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1787784728.053545:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668382720:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784728.053591:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668382720:12345-0@lo:400:kworker.0 Request processed in 46us (149us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784728.053598:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1787784728.053633:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668382720:0@lo:400:kworker.0 00000100:00100000:3.0:1787784728.054058:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894018352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668382592:0@lo:400:kworker.0 00000100:00100000:3.0:1787784728.054095:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784728.054190:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668382592 00000100:00100000:0.0:1787784728.054195:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668382592 00000100:00100000:0.0:1787784728.054230:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 538 00000100:00100000:0.0:1787784728.054237:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668382592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784728.054286:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668382592:12345-0@lo:400:kworker.0 Request processed in 50us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784728.054299:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 538 00000100:00100000:3.0:1787784728.054445:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894018352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668382592:0@lo:400:kworker.0 00000100:00100000:3.0:1787784728.056608:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668382848:0@lo:400:kworker.0 00000100:00100000:3.0:1787784728.056671:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784728.056837:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668382976:0@lo:400:kworker.0 00000100:00100000:0.0:1787784728.056872:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784728.056943:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668382848 00000100:00100000:3.0:1787784728.056950:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668382848 00000100:00100000:3.0:1787784728.057049:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:3.0:1787784728.057059:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668382848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784728.057107:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668382848:12345-0@lo:400:kworker.0 Request processed in 50us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784728.057115:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:3.0:1787784728.057175:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668382848:0@lo:400:kworker.0 00000100:00100000:1.0:1787784728.057307:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668382976 00000100:00100000:1.0:1787784728.057311:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668382976 00000100:00080000:0.0:1787784728.057858:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784728.057866:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784728.057971:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1787784728.057982:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668382976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784728.058057:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668382976:12345-0@lo:400:kworker.0 Request processed in 72us (1179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784728.058064:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1787784728.058086:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668383104:0@lo:400:kworker.0 00000100:00100000:1.0:1787784728.058115:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784728.058166:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668383104 00000100:00100000:1.0:1787784728.058170:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668383104 00000100:00080000:0.0:1787784728.058252:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784728.058471:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1787784728.058479:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668383104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784728.058522:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668383104:12345-0@lo:400:kworker.0 Request processed in 44us (407us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784728.058529:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1787784728.058678:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668383104:0@lo:400:kworker.0 00000100:00100000:1.0:1787784728.058711:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668382976:0@lo:400:kworker.0 00010000:00010000:0.0:1787784728.497400:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787784728.649649:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967341 is committed 00000001:00080000:1.0:1787784728.658052:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967892 is committed 00000001:00080000:1.0:1787784728.658069:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967887 is committed 00000020:00080000:1.0:1787784728.658089:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 14 batchid updated 00000020:00080000:0.0:1787784728.658119:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 14 00000020:00080000:0.0:1787784728.658129:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 14 00010000:00010000:2.0:1787784729.521856:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784729.647493:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967350 is committed 00010000:00010000:1.0:1787784730.547294:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784731.122725:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784731.122760:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668383232:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784731.122855:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784731.123692:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668383232 00000100:00100000:3.0:1787784731.123705:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668383232 00000100:00100000:3.0:1787784731.123797:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:3.0:1787784731.123809:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668383232:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784731.123858:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 73203712 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784731.123963:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668383232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 156us (1107us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784731.123991:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1787784731.124353:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967341 gen 2 00000004:00080000:2.0:1787784731.124361:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9894021852c0, transno 4294967341 00000100:00100000:2.0:1787784731.124388:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668383232:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784731.124543:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784731.407255:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784731.407312:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784731.407346:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784731.407371:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784731.407397:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784731.407418:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784731.407497:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784731.407540:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075392 00000100:00100000:1.0:1787784731.407550:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075392 00000100:00100000:1.0:1787784731.407604:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075904 00000100:00100000:1.0:1787784731.407607:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075904 00000100:00100000:1.0:1787784731.407625:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075520 00000100:00100000:1.0:1787784731.407627:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075520 00000100:00100000:1.0:1787784731.407638:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075776 00000100:00100000:1.0:1787784731.407641:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075776 00000100:00100000:1.0:1787784731.407656:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 288 00000100:00100000:1.0:1787784731.407665:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658075392:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784731.407735:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658075392:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 71us (422us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784731.407757:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 288 00000100:00100000:1.0:1787784731.407773:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 289 00000100:00100000:1.0:1787784731.407779:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658075904:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784731.407807:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658075904:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784731.407813:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 289 00000100:00100000:1.0:1787784731.407825:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:1.0:1787784731.407831:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658075520:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784731.407854:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658075520:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784731.407860:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:1.0:1787784731.407873:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:1.0:1787784731.407878:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658075776:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784731.407901:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658075776:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784731.407907:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:1.0:1787784731.407925:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075648 00000100:00100000:1.0:1787784731.407928:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075648 00000100:00100000:1.0:1787784731.407953:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075264 00000100:00100000:1.0:1787784731.407955:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075264 00000100:00100000:1.0:1787784731.407992:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 339 00000100:00100000:1.0:1787784731.407998:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2356:x1874627658075648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784731.408028:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2356:x1874627658075648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (682us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784731.408041:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 339 00000100:00100000:1.0:1787784731.408053:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:1.0:1787784731.408059:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:2356:x1874627658075264:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784731.408084:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:2356:x1874627658075264:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (686us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784731.408089:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00080000:0.0:1787784731.408921:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784731.408928:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784731.408930:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784731.409011:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658075136 00000100:00100000:2.0:1787784731.409019:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658075136 00000100:00100000:2.0:1787784731.409063:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 539 00000100:00100000:2.0:1787784731.409073:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658075136:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784731.409123:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658075136:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 51us (1877us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784731.409142:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 539 00010000:00010000:0.0:1787784731.571657:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784732.551924:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784732.552864:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658076032 00000100:00100000:3.0:1787784732.552879:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658076032 00000100:00100000:3.0:1787784732.552970:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 908 00000100:00100000:3.0:1787784732.552982:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:25250:x1874627658076032:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1787784732.553129:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:25250:x1874627658076032:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 148us (1214us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784732.553138:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 908 00010000:00010000:1.0:1787784732.599931:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784732.849540:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784732.849553:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784732.850021:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 2 00000040:00080000:1.0:1787784732.850033:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784733.169505:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668383360:0@lo:400:kworker.0 00000100:00100000:1.0:1787784733.169538:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668383744:0@lo:400:kworker.0 00000100:00100000:2.0:1787784733.169694:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668383488:0@lo:400:kworker.0 00000100:00100000:0.0:1787784733.169719:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668383616:0@lo:400:kworker.0 00000100:00100000:3.0:1787784733.179510:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784733.179531:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784733.179642:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668383872:0@lo:400:kworker.0 00000100:00100000:2.0:1787784733.179648:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784733.179661:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668383744 00000100:00100000:1.0:1787784733.179673:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668383744 00000100:00100000:1.0:1787784733.179743:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:1.0:1787784733.179755:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668383744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784733.180505:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668383488 00000100:00100000:0.0:1787784733.180513:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668383488 00000100:00100000:0.0:1787784733.180555:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 341 00000100:00100000:0.0:1787784733.180564:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668383488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784733.183922:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784733.183992:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668384000:0@lo:400:kworker.0 00000100:00100000:3.0:1787784733.184036:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668383360 00000100:00100000:3.0:1787784733.184048:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668383360 00000100:00100000:2.0:1787784733.184062:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784733.184125:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 540 00000100:00100000:3.0:1787784733.184140:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668383360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784733.184142:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668383872 00000100:00100000:3.0:1787784733.184230:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668383360:12345-0@lo:400:kworker.0 Request processed in 94us (4732us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784733.184237:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 540 00000100:00100000:1.0:1787784733.184279:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668383744:12345-0@lo:400:kworker.0 Request processed in 4524us (4746us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784733.184291:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:2.0:1787784733.184307:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668383872 00000100:00100000:3.0:1787784733.184312:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668383360:0@lo:400:kworker.0 00000100:00100000:0.0:1787784733.184314:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668383488:12345-0@lo:400:kworker.0 Request processed in 3746us (4666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784733.184315:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668384000 00000100:00100000:1.0:1787784733.184321:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668384000 00000100:00100000:0.0:1787784733.184330:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 341 00000100:00100000:2.0:1787784733.184364:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:1.0:1787784733.184369:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:2.0:1787784733.184377:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668383872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784733.184383:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668384000:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1787784733.184388:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784733.184396:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784733.184398:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784733.184448:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668383872:12345-0@lo:400:kworker.0 Request processed in 73us (537us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784733.184455:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668384000:12345-0@lo:400:kworker.0 Request processed in 76us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784733.184469:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:2.0:1787784733.184473:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1787784733.184524:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668383872:0@lo:400:kworker.0 00000100:00100000:0.0:1787784733.184542:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668383488:0@lo:400:kworker.0 00000100:00100000:0.0:1787784733.184567:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668384000:0@lo:400:kworker.0 00000100:00100000:0.0:1787784733.184620:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784733.184625:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668383744:0@lo:400:kworker.0 00000100:00100000:0.0:1787784733.184649:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668383616 00000100:00100000:1.0:1787784733.184662:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668384128:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784733.184668:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668383616 00000100:00100000:1.0:1787784733.184697:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784733.184707:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:0.0:1787784733.184717:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668383616:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1787784733.184739:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784733.184775:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668383616:12345-0@lo:400:kworker.0 Request processed in 60us (155us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784733.184852:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668384128 00000100:00100000:2.0:1787784733.184857:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668384128 00000100:00100000:2.0:1787784733.184894:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:2.0:1787784733.184902:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668384128:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784733.184940:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 74776576 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784733.185594:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668383616:0@lo:400:kworker.0 00000100:00100000:2.0:1787784733.185738:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668384128:12345-0@lo:13:osp-pre-1-0.0 Request processed in 835us (1039us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784733.185748:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1787784733.185832:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668384128:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784733.186838:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00010000:00010000:1.0:1787784733.618360:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787784734.155289:0:19030:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12e: Check forbidden restore ========== 18:52:12 (1787784732) 00000100:00100000:3.0:1787784734.471672:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.476137:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658076160 00000100:00100000:2.0:1787784734.476186:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658076160 00000100:00100000:2.0:1787784734.476261:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 909 00000100:00100000:2.0:1787784734.476274:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:25890:x1874627658076160:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1787784734.476389:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+37:25890:x1874627658076160:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 119us (4725us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.476397:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 909 00000100:00080000:0.0:1787784734.478067:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784734.536983:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.537217:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658076288 00000100:00100000:3.0:1787784734.537229:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658076288 00000100:00100000:3.0:1787784734.537344:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 910 00000100:00100000:3.0:1787784734.537356:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25891:x1874627658076288:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784734.537382:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784734.537411:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57c6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ce1 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537499:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537517:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537535:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537552:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537559:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537567:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537885:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537894:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537901:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537917:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.537923:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.537931:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784734.538192:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784734.538435:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:3.0:1787784734.538440:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.538446:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538454:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc57cd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538472:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538478:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538486:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538491:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.538495:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538502:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538525:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff989505542d00) 00010000:00010000:3.0:1787784734.538527:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.538531:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538537:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57d4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538584:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57c6 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ce1 expref: 83 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538596:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:3.0:1787784734.538599:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.538603:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57c6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ce1 expref: 83 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.538613:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784734.538698:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25891:x1874627658076288:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1344us (1730us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784734.538708:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 910 00000100:00100000:2.0:1787784734.545758:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.545860:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658076544 00000100:00100000:3.0:1787784734.545872:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658076544 00000100:00100000:3.0:1787784734.545962:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 911 00000100:00100000:3.0:1787784734.545974:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25891:x1874627658076544:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:3.0:1787784734.546051:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784734.546101:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5637 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c40 expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.546194:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:3.0:1787784734.546201:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784734.546539:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.546555:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 36 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.546566:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.546608:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 36 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.546625:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 36 pid: 5812 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784734.546647:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668384256:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784734.546766:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784734.549923:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.550483:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668384256:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784734.550521:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784734.551039:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658076672 00000100:00100000:1.0:1787784734.551048:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658076672 00010000:00010000:1.0:1787784734.551220:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 37 pid: 5812 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784734.551316:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00100000:1.0:1787784734.551328:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:2357:x1874627658076672:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784734.551346:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784734.551353:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 37 pid: 5812 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.551368:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 37 pid: 5812 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.551416:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc5676 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0c55 expref: 37 pid: 5812 timeout: 625 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.551437:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:1.0:1787784734.551442:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.551452:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.551461:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.551494:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784734.551585:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+36:2357:x1874627658076672:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 259us (1677us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784734.551595:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 280 00010000:00010000:3.0:1787784734.552471:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.552485:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.552540:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.552554:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.552565:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784734.553930:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 512 00010000:00010000:3.0:1787784734.553964:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.553977:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.553984:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.553992:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.553996:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.554005:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.554022:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407a00) 00010000:00010000:3.0:1787784734.554025:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.554030:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.554049:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc57db lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784734.554077:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008dac85b4 reply state 000000005bacf707 transno 4294967893 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784734.554087:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784734.554264:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25891:x1874627658076544:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 8287us (8515us total) trans 4294967893 rc 0/0 00000100:00100000:3.0:1787784734.554276:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 911 00010000:00010000:2.0:1787784734.558973:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.558996:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.559004:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.559034:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784734.559040:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.559048:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.559307:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff989505542d00) 00010000:00010000:2.0:1787784734.559311:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784734.559318:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.559326:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc57e2 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784734.572327:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784734.572550:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658076800 00000100:00100000:0.0:1787784734.572563:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658076800 00000100:00100000:0.0:1787784734.572638:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 912 00000100:00100000:0.0:1787784734.572650:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+82:25892:x1874627658076800:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784734.572692:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784734.572725:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc57e9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ce8 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.572812:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc57f0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.572830:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc57f0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.572844:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc57f0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784734.572927:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784734.572957:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 24 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784734.572992:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc57f0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.573022:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc57e9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ce8 expref: 84 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.573045:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc57f0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ce8 expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.573054:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784734.573058:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784734.573062:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030200, rc 0) 00000100:00100000:0.0:1787784734.573142:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25892:x1874627658076800:12345-192.168.203.43@tcp:101:bash.0 Request processed in 495us (829us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784734.573215:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 912 00000100:00100000:0.0:1787784734.579062:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.579186:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658076928 00000100:00100000:1.0:1787784734.579199:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658076928 00000100:00100000:1.0:1787784734.579267:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 913 00000100:00100000:1.0:1787784734.579276:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+83:25892:x1874627658076928:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784734.579303:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784734.579328:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc57f7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cef expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.579407:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.579449:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.579462:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.579482:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.579490:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.579497:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580127:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1dc00/0xb0ce63137acc580c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580142:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1dc00/0xb0ce63137acc580c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580190:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1dc00/0xb0ce63137acc580c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784734.580233:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 512 00000004:00200000:1.0:1787784734.580238:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784734.580267:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1dc00/0xb0ce63137acc580c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580283:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580303:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580309:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580315:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784734.580321:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580330:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580346:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:1.0:1787784734.580349:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.580354:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580360:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc57fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580377:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580382:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580388:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580393:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784734.580401:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580407:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580414:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff98951c407b00) 00010000:00010000:1.0:1787784734.580417:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.580420:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580426:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5805 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580468:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc57f7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cef expref: 85 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580483:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1dc00/0xb0ce63137acc580c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cef expref: 84 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.580491:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:1.0:1787784734.580493:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784734.580496:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1dc00, rc 0) 00000100:00100000:1.0:1787784734.580604:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25892:x1874627658076928:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1328us (1555us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784734.580614:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 913 00000100:00100000:2.0:1787784734.589335:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784734.589843:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658077056 00000100:00100000:0.0:1787784734.589857:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658077056 00000100:00100000:0.0:1787784734.589930:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 914 00000100:00100000:0.0:1787784734.589954:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25892:x1874627658077056:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784734.589983:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784734.590011:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5813 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cf6 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590117:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590136:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590223:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590242:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590249:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590254:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590349:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590358:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590365:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590372:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784734.590377:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590385:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590398:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:0.0:1787784734.590401:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784734.590405:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590418:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc581a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590433:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590440:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590446:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590452:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784734.590455:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590461:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590469:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff989503a6de00) 00010000:00010000:0.0:1787784734.590471:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784734.590474:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590479:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5821 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590522:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5813 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cf6 expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590560:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:0.0:1787784734.590563:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784734.590567:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5813 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cf6 expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.590578:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030c00, rc 0) 00000100:00100000:0.0:1787784734.590664:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25892:x1874627658077056:12345-192.168.203.43@tcp:101:bash.0 Request processed in 724us (1342us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784734.590675:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 914 00000100:00100000:2.0:1787784734.607743:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.607908:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658077184 00000100:00100000:1.0:1787784734.607924:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658077184 00000100:00100000:1.0:1787784734.608013:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 915 00000100:00100000:1.0:1787784734.608024:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+36:25893:x1874627658077184:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784734.608084:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784734.608114:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5828 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0cfd expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.608256:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.608275:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.608289:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784734.608374:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784734.608379:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 24 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784734.608408:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.608438:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5828 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 38 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.608461:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 37 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.608470:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:1.0:1787784734.608473:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784734.608477:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d600, rc 0) 00000100:00100000:1.0:1787784734.608626:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+37:25893:x1874627658077184:12345-192.168.203.43@tcp:101:rm.0 Request processed in 603us (898us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784734.608633:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 915 00000100:00100000:1.0:1787784734.618558:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784734.618726:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658077312 00000100:00100000:0.0:1787784734.618741:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658077312 00000100:00100000:0.0:1787784734.618813:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 916 00000100:00100000:0.0:1787784734.618825:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+37:25893:x1874627658077312:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784734.618854:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784734.618880:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5836 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d04 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.618983:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619001:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619014:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619032:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619058:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619065:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619256:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030400/0xb0ce63137acc584b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619266:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030400/0xb0ce63137acc584b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619273:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030400/0xb0ce63137acc584b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784734.619311:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 512 00000004:00200000:0.0:1787784734.619314:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784734.619340:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500030400/0xb0ce63137acc584b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619355:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619363:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619369:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619375:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784734.619379:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619395:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619410:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784734.619414:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784734.619419:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619426:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc583d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619444:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619450:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619455:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619460:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784734.619462:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619468:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619475:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff989503a6de00) 00010000:00010000:0.0:1787784734.619478:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784734.619481:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619497:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5844 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619643:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5836 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d04 expref: 39 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619677:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030400/0xb0ce63137acc584b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d04 expref: 38 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.619686:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:0.0:1787784734.619689:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784734.619692:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030400, rc 0) 00000100:00100000:0.0:1787784734.619832:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25893:x1874627658077312:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1006us (1285us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784734.619841:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 916 00000100:00100000:2.0:1787784734.622840:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.622961:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658077440 00000100:00100000:1.0:1787784734.622974:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658077440 00000100:00100000:1.0:1787784734.623044:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 917 00000100:00100000:1.0:1787784734.623055:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25893:x1874627658077440:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784734.623083:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784734.623109:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5852 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d0b expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623226:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623244:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623258:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623273:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623280:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623286:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623367:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623374:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623398:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623404:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784734.623409:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623417:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623431:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:1.0:1787784734.623434:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.623439:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623445:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5859 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623460:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623466:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623472:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623477:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784734.623481:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623498:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623507:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff98951c407b00) 00010000:00010000:1.0:1787784734.623509:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.623512:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623517:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5860 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623559:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5852 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d0b expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623570:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:1.0:1787784734.623572:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.623574:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5852 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d0b expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.623583:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784734.623661:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25893:x1874627658077440:12345-192.168.203.43@tcp:101:rm.0 Request processed in 607us (833us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784734.623669:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 917 00000100:00100000:2.0:1787784734.632049:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.632179:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658077568 00000100:00100000:3.0:1787784734.632194:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658077568 00000100:00100000:3.0:1787784734.632270:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 918 00000100:00100000:3.0:1787784734.632283:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25893:x1874627658077568:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784734.632310:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784734.632338:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5867 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d12 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632426:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632443:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632457:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632473:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632506:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632513:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632617:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632626:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632634:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632641:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.632646:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632654:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632669:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:3.0:1787784734.632672:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.632677:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632684:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc586e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632702:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632715:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632722:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632727:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.632730:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632737:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632746:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff989505542600) 00010000:00010000:3.0:1787784734.632749:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.632752:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632759:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5875 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632800:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5867 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d12 expref: 39 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632812:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:3.0:1787784734.632814:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.632818:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5867 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d12 expref: 39 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.632841:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784734.632924:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25893:x1874627658077568:12345-192.168.203.43@tcp:101:rm.0 Request processed in 643us (890us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784734.632933:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 918 00010000:00010000:0.0:1787784734.642912:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784734.658110:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.659891:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658077696 00000100:00100000:1.0:1787784734.659904:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658077696 00000100:00100000:1.0:1787784734.660037:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 919 00000100:00100000:1.0:1787784734.660048:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25894:x1874627658077696:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784734.661510:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784734.661533:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc587c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d19 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661607:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661620:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661633:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661647:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661653:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661659:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661731:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661738:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661744:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661750:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784734.661767:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661776:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661788:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:1.0:1787784734.661791:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.661795:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661801:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5883 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661815:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661821:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661828:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661833:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784734.661835:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661841:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661848:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff98951c407b00) 00010000:00010000:1.0:1787784734.661850:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.661860:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661867:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc588a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661904:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc587c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d19 expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661914:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:1.0:1787784734.661916:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.661920:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc587c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d19 expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.661930:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784734.662017:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25894:x1874627658077696:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1971us (3916us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784734.662026:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 919 00000100:00100000:2.0:1787784734.664078:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.664188:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658077824 00000100:00100000:3.0:1787784734.664197:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658077824 00000100:00100000:3.0:1787784734.664393:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 920 00000100:00100000:3.0:1787784734.664410:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25894:x1874627658077824:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784734.664433:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784734.664458:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5891 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d20 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664519:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664532:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664552:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664568:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664575:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664583:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664665:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664673:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664678:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664692:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.664697:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664709:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664724:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:3.0:1787784734.664727:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.664734:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664740:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5898 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664759:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664765:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664777:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664791:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.664797:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664804:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664814:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff989505542600) 00010000:00010000:3.0:1787784734.664819:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.664823:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664829:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc589f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664871:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5891 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d20 expref: 39 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664886:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:3.0:1787784734.664889:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.664892:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5891 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d20 expref: 39 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.664907:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784734.664981:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25894:x1874627658077824:12345-192.168.203.43@tcp:101:cp.0 Request processed in 579us (916us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784734.664992:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 920 00000100:00100000:2.0:1787784734.675471:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.675566:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658078080 00000100:00100000:3.0:1787784734.675588:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658078080 00000100:00100000:3.0:1787784734.675648:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 921 00000100:00100000:3.0:1787784734.675659:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:25894:x1874627658078080:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784734.675679:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784734.675697:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc58a6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d27 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.675963:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.675979:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.675990:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.676005:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.676012:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.676018:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784734.676832:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x12:0x0], index 0: 0 00010000:00010000:3.0:1787784734.677024:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7a00/0xb0ce63137acc58bb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677038:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7a00/0xb0ce63137acc58bb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677057:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7a00/0xb0ce63137acc58bb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677093:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677101:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677107:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677113:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784734.677117:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677124:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.677137:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:3.0:1787784734.677140:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784734.677145:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784734.678370:0:6996:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304a580 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6996:1874627668384384:0@lo:5:osp-pre-0-0.0 00010000:00010000:3.0:1787784734.678563:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc58ad lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784734.678584:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a0d6ce32 reply state 000000008023be2e transno 4294967894 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784734.678592:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784734.678932:0:6996:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784734.679019:0:6996:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784734.679088:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668384384 00000100:00100000:0.0:1787784734.679096:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668384384 00000100:00100000:0.0:1787784734.679175:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:0.0:1787784734.679186:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:6996:x1874627668384384:12345-0@lo:5:osp-pre-0-0.0 00002000:00080000:0.0:1787784734.679227:0:6671:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0000: reserve 32 objects in group 0x240000400 at 34 00010000:00010000:2.0:1787784734.679454:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc58a6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d27 expref: 41 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.679474:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7a00/0xb0ce63137acc58bb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d27 expref: 40 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.679505:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:2.0:1787784734.679508:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784734.679511:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7a00, rc 0) 00000100:00100000:2.0:1787784734.679586:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25894:x1874627658078080:12345-192.168.203.43@tcp:101:cp.0 Request processed in 3929us (4124us total) trans 4294967894 rc 0/0 00000100:00100000:2.0:1787784734.679595:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 921 00010000:00010000:1.0:1787784734.682883:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.682900:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.682907:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.682914:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784734.682918:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.682952:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.682969:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff989505542600) 00010000:00010000:1.0:1787784734.682972:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784734.682977:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.682983:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc58b4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784734.683267:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.683455:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658078208 00000100:00100000:2.0:1787784734.683467:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658078208 00000100:00080000:1.0:1787784734.683554:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784734.683559:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 235 00000100:00100000:2.0:1787784734.683570:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658078208:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784734.683586:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784734.683605:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d2e expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784734.683628:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x12:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784734.683648:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=525s v=(0 0) 00010000:00010000:2.0:1787784734.683653:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784734.683662:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.683678:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.683691:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784734.683758:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658078208:12345-192.168.203.43@tcp:101:cp.0 Request processed in 190us (504us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.683775:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 235 00000100:00100000:3.0:1787784734.689275:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.689514:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658078336 00000100:00100000:1.0:1787784734.689527:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658078336 00000100:00100000:1.0:1787784734.689594:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 236 00000100:00100000:1.0:1787784734.689604:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:25894:x1874627658078336:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784734.689621:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784734.689639:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d35 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.689668:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.689678:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784734.689682:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784734.689688:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.689732:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.689747:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 10 pid: 6670 timeout: 625 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784734.689761:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305043c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668384512:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784734.689842:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953079a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784734.693993:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:6996:x1874627668384384:12345-0@lo:5:osp-pre-0-0.0 Request processed in 14803us (15055us total) trans 4294967342 rc 0/0 00000100:00100000:2.0:1787784734.694007:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1787784734.698774:0:6996:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304a580 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6996:1874627668384384:0@lo:5:osp-pre-0-0.0 00000004:00080000:3.0:1787784734.698803:0:6996:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0000-osc-MDT0000: current precreated pool: [0x240000400:0x12:0x0]-[0x240000400:0x41:0x0] 00000100:00100000:1.0:1787784734.700824:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.701030:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305043c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668384512:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784734.701212:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.701240:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:2.0:1787784734.701309:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:25894:x1874627658078336:12345-192.168.203.43@tcp:101:cp.0 Request processed in 11706us (12051us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.701320:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 236 00000100:00100000:0.0:1787784734.701408:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658078464 00000100:00100000:0.0:1787784734.701422:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658078464 00010000:00010000:0.0:1787784734.701467:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 10 pid: 6670 timeout: 625 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784734.701527:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 281 00000100:00100000:0.0:1787784734.701554:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658078464:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784734.701571:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784734.701603:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 10 pid: 6670 timeout: 625 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.701627:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 10 pid: 6670 timeout: 625 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.701650:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f035e1e6/0xb0ce63137acc58c9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d2e expref: 10 pid: 6670 timeout: 625 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.701668:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784734.701680:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784734.701690:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 9 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.701701:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 9 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.701739:0:11303:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 9 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784734.701754:0:11303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304a940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668384640:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784734.701832:0:11303:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784734.717851:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.717991:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658078720 00000100:00100000:3.0:1787784734.718001:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658078720 00000100:00100000:3.0:1787784734.718061:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 186 00000100:00100000:3.0:1787784734.718071:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:25894:x1874627658078720:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:3.0:1787784734.718441:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25894:x1874627658078720:12345-192.168.203.43@tcp:35:cp.0 Request processed in 370us (597us total) trans 4294967895 rc 0/0 00000100:00100000:3.0:1787784734.718450:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 186 00000100:00100000:2.0:1787784734.719417:0:11303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304a940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668384640:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784734.719441:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784734.719483:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658078464:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 17933us (18673us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.719493:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 281 00000100:00100000:0.0:1787784734.781790:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.781996:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658078848 00000100:00100000:2.0:1787784734.782006:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658078848 00000100:00100000:2.0:1787784734.782403:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 922 00000100:00100000:2.0:1787784734.782416:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:25897:x1874627658078848:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784734.782436:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784734.782457:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc58d7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d3c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782543:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782557:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782567:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782581:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782589:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782596:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782693:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc58ec lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782703:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc58ec lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782709:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc58ec lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784734.782741:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 158 00000004:00200000:2.0:1787784734.782745:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784734.782763:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc58ec lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782785:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc58ec lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782794:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782801:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782806:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782812:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784734.782816:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782823:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782834:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:2.0:1787784734.782837:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784734.782841:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782847:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc58de lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782859:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782865:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782878:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782884:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784734.782887:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782893:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782901:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff989530cb1700) 00010000:00010000:2.0:1787784734.782904:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784734.782906:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782912:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc58e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782953:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc58d7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d3c expref: 86 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782966:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc58ec lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d3c expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.782974:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:2.0:1787784734.782976:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784734.782979:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784734.783673:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:25897:x1874627658078848:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 634us (1266us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.783696:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 922 00000100:00100000:3.0:1787784734.789451:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.789542:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658078976 00000100:00100000:2.0:1787784734.789551:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658078976 00000100:00100000:2.0:1787784734.789619:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 237 00000100:00100000:2.0:1787784734.789629:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658078976:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784734.789646:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784734.789662:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc58f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d43 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784734.789719:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895cf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668384768:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784734.789797:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d840 going to sleep for 11 seconds 00000100:00080000:1.0:1787784734.789859:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1787784734.796265:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784734.796276:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb mtime: 0 -> 1787784734 00002000:00010000:2.0:1787784734.796280:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb atime: 0 -> 1787784734 00002000:00010000:2.0:1787784734.796282:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb ctime: 0 -> 1787784734 00000100:00100000:2.0:1787784734.796305:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895cf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668384768:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784734.796342:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc58f3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d43 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.796367:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc58f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d43 expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.796382:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784734.796447:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658078976:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6819us (7004us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784734.796479:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 237 00000100:00100000:0.0:1787784734.806050:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.806185:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658079104 00000100:00100000:1.0:1787784734.806195:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658079104 00000100:00100000:1.0:1787784734.806259:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 923 00000100:00100000:1.0:1787784734.806269:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:25897:x1874627658079104:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784734.806288:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784734.806306:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc58fa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d4a expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.806369:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5901 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.806382:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5901 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.806393:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5901 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784734.806466:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 512 00000004:00200000:1.0:1787784734.806470:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784734.806492:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5901 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.806516:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc58fa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d4a expref: 87 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.806545:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5901 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d4a expref: 86 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.806554:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:1.0:1787784734.806557:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784734.806560:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d200, rc 0) 00000100:00100000:1.0:1787784734.806620:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:25897:x1874627658079104:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 353us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784734.806628:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 923 00000100:00100000:0.0:1787784734.812822:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.813256:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658079232 00000100:00100000:3.0:1787784734.813267:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658079232 00000100:00100000:3.0:1787784734.813333:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 238 00000100:00100000:3.0:1787784734.813343:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658079232:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784734.813359:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784734.813375:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5908 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d51 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784734.813436:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668384896:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784734.813521:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643780 going to sleep for 11 seconds 00000100:00100000:1.0:1787784734.820868:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668384896:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784734.820909:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5908 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d51 expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.820949:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5908 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d51 expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.820966:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:1.0:1787784734.821028:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2357:x1874627658079232:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7686us (8216us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784734.821062:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 238 00000100:00100000:0.0:1787784734.841533:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.841609:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658079488 00000100:00100000:3.0:1787784734.841620:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658079488 00000100:00100000:3.0:1787784734.841680:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 924 00000100:00100000:3.0:1787784734.841690:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:25897:x1874627658079488:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784734.842003:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].23, 136 off11320 00000100:00100000:3.0:1787784734.842100:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:25897:x1874627658079488:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 402us (569us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784734.842110:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 924 00000040:00080000:1.0:1787784734.898386:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784734.898395:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784734.898434:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 24 count 3 00000040:00080000:1.0:1787784734.898445:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784734.898692:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668385024:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784734.899278:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953079a600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784734.911455:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668385024:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784734.917369:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.917650:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658079616 00000100:00100000:3.0:1787784734.917664:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658079616 00000100:00100000:3.0:1787784734.917751:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 925 00000100:00100000:3.0:1787784734.917763:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25903:x1874627658079616:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784734.917819:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x18:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784734.918363:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25903:x1874627658079616:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 600us (1006us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784734.918374:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 925 00000100:00100000:0.0:1787784734.931186:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.931253:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658079872 00000100:00100000:2.0:1787784734.931267:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658079872 00000100:00100000:2.0:1787784734.931341:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 926 00000100:00100000:2.0:1787784734.931352:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:25903:x1874627658079872:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784734.931372:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784734.931393:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc590f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d58 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.931495:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5916 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.931509:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5916 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.931523:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5916 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784734.932013:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 158 00010000:00010000:2.0:1787784734.932222:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc590f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d58 expref: 44 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.932243:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5916 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d58 expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.932252:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:2.0:1787784734.932255:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784734.932259:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc800, rc 0) 00000100:00100000:2.0:1787784734.932335:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:25903:x1874627658079872:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 984us (1192us total) trans 4294967896 rc 0/0 00000100:00100000:2.0:1787784734.932343:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 926 00000100:00100000:2.0:1787784734.936724:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.936984:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658080000 00000100:00100000:1.0:1787784734.936994:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658080000 00000100:00100000:1.0:1787784734.937054:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00100000:1.0:1787784734.937064:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658080000:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784734.937106:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.937127:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 9 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.937136:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x12:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1787784734.937140:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784734.937145:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.937227:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 9 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.937255:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 9 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784734.937270:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530507480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668385152:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784734.937356:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953079a600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784734.940435:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784734.940643:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658080128 00000100:00100000:0.0:1787784734.940653:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658080128 00000100:00100000:0.0:1787784734.940724:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 38 00000100:00100000:0.0:1787784734.940739:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2354:x1874627658080128:12345-192.168.203.43@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787784734.940742:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:0.0:1787784734.940758:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787784734.940775:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658080256 00000100:00100000:3.0:1787784734.940780:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658080256 00010000:00010000:0.0:1787784734.940786:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5916 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d58 expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784734.940807:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:0.0:1787784734.940812:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00100000:3.0:1787784734.940816:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (0->157) req@ffff989506334f00 x1874627658080256/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:615/0 lens 648/0 e 0 to 0 dl 1787784745 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784734.940828:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989506334f00 with x1874627658080256 ext(0->157) 00010000:00010000:3.0:1787784734.940837:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 10 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784734.940856:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 10 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784734.940867:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:2354:x1874627658080128:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 131us (438us total) trans 0 rc 0/0 00002000:00010000:3.0:1787784734.940869:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989506334f00 00000100:00100000:0.0:1787784734.940884:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 38 00000100:00100000:3.0:1787784734.940894:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 8 00000100:00100000:3.0:1787784734.940917:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658080256:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784734.941022:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530507480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668385152:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784734.941048:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784734.943430:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658080256:12345-192.168.203.43@tcp:4:cp.0 Request processed in 2505us (2684us total) trans 4294967343 rc 0/0 00000100:00100000:2.0:1787784734.943443:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 8 00002000:00100000:2.0:1787784734.943455:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (0->157) req@ffff989506334f00 x1874627658080256/t4294967343(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:615/0 lens 648/448 e 0 to 0 dl 1787784745 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784734.943464:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989506334f00 with x1874627658080256 ext(0->157) 00010000:00010000:2.0:1787784734.943473:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 12 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.943491:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 12 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784734.943499:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989506334f00 00000100:00100000:0.0:1787784734.946650:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.946731:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658080384 00000100:00100000:2.0:1787784734.946741:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658080384 00010000:00010000:2.0:1787784734.946780:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 12 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784734.946846:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 282 00000100:00100000:2.0:1787784734.946857:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895057ec850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658080384:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784734.946873:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784734.946894:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 12 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.946909:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 12 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.946937:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc58d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d35 expref: 12 pid: 10688 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.946968:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784734.946973:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784734.946979:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.946990:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.947026:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784734.947078:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895057ec850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658080384:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 223us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.947087:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 282 00010000:00010000:1.0:1787784734.947402:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947414:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947439:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947448:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947456:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947462:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784734.947467:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947488:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947500:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784734.947507:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b8eb748/0xb0ce63137acc5924 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784734.947604:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658080000:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 10541us (10887us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784734.947613:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00100000:0.0:1787784734.951439:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.951510:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658080640 00000100:00100000:2.0:1787784734.951519:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658080640 00000100:00100000:2.0:1787784734.951578:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 927 00000100:00100000:2.0:1787784734.951588:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:25903:x1874627658080640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784734.951701:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:25903:x1874627658080640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 114us (271us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.951709:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 927 00000100:00100000:3.0:1787784734.956909:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784734.957022:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658080768 00000100:00100000:2.0:1787784734.957033:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658080768 00000100:00100000:2.0:1787784734.957095:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:2.0:1787784734.957107:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658080768:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784734.957126:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784734.957319:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d5f expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784734.957348:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x12:0x0] initial LVB size: 158, mtime: 0x6a8f6e1e, atime: 0x0, ctime: 0x6a8f6e1e, blocks: 0x1 00010000:00010000:2.0:1787784734.957371:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=526s v=(0 0) 00010000:00010000:2.0:1787784734.957375:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784734.957385:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.957401:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784734.957414:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784734.957511:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658080768:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 407us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784734.957541:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:0.0:1787784734.970296:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784734.970583:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658081024 00000100:00100000:3.0:1787784734.970592:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658081024 00000100:00100000:3.0:1787784734.970690:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 928 00000100:00100000:3.0:1787784734.970701:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:25903:x1874627658081024:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784734.970818:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:25903:x1874627658081024:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 118us (531us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784734.970827:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 928 00000100:00100000:0.0:1787784734.990276:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784734.990389:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658081152 00000100:00100000:1.0:1787784734.990399:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658081152 00002000:00100000:1.0:1787784734.990462:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x12:0x0] (0->4095) req@ffff9894021843c0 x1874627658081152/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:615/0 lens 488/0 e 0 to 0 dl 1787784745 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784734.990474:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894021843c0 with x1874627658081152 ext(0->4095) 00002000:00010000:1.0:1787784734.990479:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894021843c0 00000100:00100000:1.0:1787784734.990508:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 87 00000100:00100000:1.0:1787784734.990518:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658081152:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1787784734.990710:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658081152:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 193us (444us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787784734.990719:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 87 00002000:00100000:1.0:1787784734.990740:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x12:0x0] (0->4095) req@ffff9894021843c0 x1874627658081152/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:615/0 lens 488/4536 e 0 to 0 dl 1787784745 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784734.990749:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894021843c0 with x1874627658081152 ext(0->4095) 00002000:00010000:1.0:1787784734.990752:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894021843c0 00000100:00100000:3.0:1787784735.005253:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784735.007667:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658081280 00000100:00100000:0.0:1787784735.007682:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658081280 00000100:00100000:0.0:1787784735.007748:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 929 00000100:00100000:0.0:1787784735.007773:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:25903:x1874627658081280:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784735.007794:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784735.007816:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5932 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d66 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.007851:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.007863:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.007873:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.007984:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5932 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.008000:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 43 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.008009:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:0.0:1787784735.008012:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784735.008015:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030c00, rc 0) 00000100:00100000:0.0:1787784735.008085:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:25903:x1874627658081280:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 316us (2841us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784735.008092:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 929 00000100:00100000:0.0:1787784735.016223:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.016302:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658081408 00000100:00100000:1.0:1787784735.016311:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658081408 00000100:00100000:1.0:1787784735.016357:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 930 00000100:00100000:1.0:1787784735.016367:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:25903:x1874627658081408:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784735.016397:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x18:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784735.016515:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:25903:x1874627658081408:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 149us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784735.016801:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 930 00000100:00100000:1.0:1787784735.023470:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784735.027800:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658081536 00000100:00100000:3.0:1787784735.027814:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658081536 00000100:00100000:3.0:1787784735.027907:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 241 00000100:00100000:3.0:1787784735.027921:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658081536:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784735.027981:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.027998:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784735.028003:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784735.028009:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028026:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028058:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028074:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028083:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028091:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784735.028096:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028106:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028123:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.028131:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c38e150/0xb0ce63137acc5940 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784735.028221:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658081536:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 304us (4760us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784735.028234:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 241 00000100:00100000:0.0:1787784735.033227:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784735.033706:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658081792 00000100:00100000:3.0:1787784735.033716:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658081792 00000100:00100000:3.0:1787784735.033763:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 931 00000100:00100000:3.0:1787784735.033787:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:25903:x1874627658081792:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784735.034130:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.034168:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 43 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.034177:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.034212:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 43 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.034226:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 43 pid: 5813 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784735.034240:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894018352c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668385280:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784735.034389:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784735.037535:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784735.037699:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658081920 00000100:00100000:2.0:1787784735.037708:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658081920 00010000:00010000:2.0:1787784735.037747:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 44 pid: 5813 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784735.037794:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 283 00000100:00100000:2.0:1787784735.037804:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2356:x1874627658081920:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784735.037820:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784735.037828:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 44 pid: 5813 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.037842:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 44 pid: 5813 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.037864:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:2.0:1787784735.037869:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.037879:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.037887:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.037906:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784735.037978:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2356:x1874627658081920:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 175us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784735.038003:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 283 00010000:00010000:1.0:1787784735.038360:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5939 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d66 expref: 43 pid: 5813 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784735.038377:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894018352c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668385280:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784735.038398:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038406:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038412:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038593:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038608:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038615:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038621:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784735.038625:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038633:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038643:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:1.0:1787784735.038646:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784735.038650:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.038656:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc5947 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784735.038796:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:25903:x1874627658081792:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 5011us (5577us total) trans 4294967897 rc 0/0 00000100:00100000:1.0:1787784735.038804:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 931 00000040:00080000:2.0:1787784735.038898:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784735.038904:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784735.039415:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1787784735.039423:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784735.042296:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784735.042377:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658082176 00000100:00100000:2.0:1787784735.042386:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658082176 00000100:00100000:2.0:1787784735.042440:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 187 00000100:00100000:2.0:1787784735.042451:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:25903:x1874627658082176:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784735.042714:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:25903:x1874627658082176:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 265us (427us total) trans 4294967898 rc 0/0 00000100:00100000:2.0:1787784735.042723:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 187 00000040:00080000:2.0:1787784735.652590:0:19080:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784735.652601:0:19080:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784735.652644:0:19080:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 24 count 3 00000040:00080000:2.0:1787784735.652650:0:19080:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787784735.682048:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784735.800196:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.801141:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658082432 00000100:00100000:1.0:1787784735.801178:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658082432 00000100:00100000:1.0:1787784735.801243:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 932 00000100:00100000:1.0:1787784735.801253:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:25908:x1874627658082432:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784735.801299:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784735.801302:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787784735.801330:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5901 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d4a expref: 86 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.801351:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:1.0:1787784735.801356:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784735.801369:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc594e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d6d expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.801462:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5955 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.801474:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5955 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.801484:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5955 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784735.801507:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 158 00010000:00010000:1.0:1787784735.802043:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc594e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d6d expref: 88 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.802060:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5955 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d6d expref: 87 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.802070:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:1.0:1787784735.802073:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784735.802076:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784735.802218:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:25908:x1874627658082432:12345-192.168.203.43@tcp:101:bash.0 Request processed in 966us (2083us total) trans 4294967899 rc 0/0 00000100:00100000:1.0:1787784735.802226:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 932 00000100:00100000:0.0:1787784735.807974:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.808405:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658082560 00000100:00100000:1.0:1787784735.808417:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658082560 00000100:00100000:1.0:1787784735.809223:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 39 00000100:00100000:1.0:1787784735.809248:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:2354:x1874627658082560:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784735.809265:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784735.809304:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5955 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d6d expref: 87 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.809324:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:1.0:1787784735.809329:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784735.809398:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:2354:x1874627658082560:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 159us (1436us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784735.815056:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 39 00000100:00100000:3.0:1787784735.823840:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784735.823966:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658082688 00000100:00100000:2.0:1787784735.823981:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658082688 00000100:00100000:2.0:1787784735.824051:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 242 00000100:00100000:2.0:1787784735.824061:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+11:2356:x1874627658082688:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784735.824079:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784735.824095:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d74 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.824121:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784735.824126:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784735.824163:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.824182:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.824195:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784735.824256:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658082688:12345-192.168.203.43@tcp:101:cat.0 Request processed in 196us (431us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784735.824264:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 242 00000100:00100000:3.0:1787784735.839489:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784735.839607:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658082816 00000100:00100000:2.0:1787784735.839624:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658082816 00000100:00100000:2.0:1787784735.839681:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 243 00000100:00100000:2.0:1787784735.839691:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:25908:x1874627658082816:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784735.839710:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784735.839728:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9b5213a/0xb0ce63137acc596a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d7b expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.839772:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.839783:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.839791:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784735.839796:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784735.839813:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9b5213a/0xb0ce63137acc596a lrc: 5/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d7b expref: 13 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.839847:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.839862:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 13 pid: 10688 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784735.839875:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895d680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668385408:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784735.839987:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784735.839998:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 10 pid: 6670 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784735.840008:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668385536:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784735.840033:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784735.848750:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.848845:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895d680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668385408:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784735.849191:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668385536:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784735.849200:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658082944 00000100:00100000:3.0:1787784735.849212:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658082944 00010000:00010000:1.0:1787784735.849233:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c9b5213a/0xb0ce63137acc596a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d7b expref: 14 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.849257:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fde00, rc 0) 00010000:00010000:3.0:1787784735.849264:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 14 pid: 10688 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784735.849331:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:25908:x1874627658082816:12345-192.168.203.43@tcp:101:cat.0 Request processed in 9637us (9854us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784735.849343:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 243 00000100:00100000:3.0:1787784735.849451:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00100000:3.0:1787784735.849467:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658082944:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784735.849486:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784735.849517:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 13 pid: 10688 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.849533:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 13 pid: 10688 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.849562:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc5963 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d74 expref: 13 pid: 10688 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.849580:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784735.849623:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658082944:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 162us (898us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784735.849643:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00100000:3.0:1787784735.854020:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.854907:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658083072 00000100:00100000:1.0:1787784735.854921:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658083072 00010000:00010000:1.0:1787784735.854975:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 11 pid: 6670 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784735.855026:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:1.0:1787784735.855035:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658083072:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784735.855049:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784735.855070:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 11 pid: 6670 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.855084:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 11 pid: 6670 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.855107:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc592b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d5f expref: 11 pid: 6670 timeout: 626 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.855123:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784735.855128:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784735.855135:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c9b5213a/0xb0ce63137acc596a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d7b expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.855169:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9b5213a/0xb0ce63137acc596a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d7b expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784735.855204:0:11303:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c9b5213a/0xb0ce63137acc596a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d7b expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784735.855226:0:11303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668385664:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784735.855310:0:11303:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784735.861238:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.861623:0:11303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668385664:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784735.861649:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784735.861701:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658083200 00000100:00100000:3.0:1787784735.861716:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658083200 00000100:00100000:1.0:1787784735.861727:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658083072:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 6691us (7720us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784735.861741:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 285 00002000:00100000:3.0:1787784735.863627:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (0->4095) req@ffff98954028da40 x1874627658083200/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:616/0 lens 488/0 e 0 to 0 dl 1787784746 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784735.863643:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954028da40 with x1874627658083200 ext(0->4095) 00002000:00010000:3.0:1787784735.863650:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98954028da40 00000100:00100000:3.0:1787784735.863756:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 88 00000100:00100000:3.0:1787784735.863788:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658083200:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:3.0:1787784735.864083:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658083200:12345-192.168.203.43@tcp:3:cat.0 Request processed in 299us (2853us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784735.864091:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 88 00002000:00100000:3.0:1787784735.864099:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (0->4095) req@ffff98954028da40 x1874627658083200/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:616/0 lens 488/4536 e 0 to 0 dl 1787784746 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784735.864108:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954028da40 with x1874627658083200 ext(0->4095) 00002000:00010000:3.0:1787784735.864112:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98954028da40 00000100:00100000:3.0:1787784735.883064:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.883199:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658083456 00000100:00100000:1.0:1787784735.883217:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658083456 00000100:00100000:1.0:1787784735.883441:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 188 00000100:00100000:1.0:1787784735.883455:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:25908:x1874627658083456:12345-192.168.203.43@tcp:35:cat.0 00000001:00080000:1.0:1787784735.883833:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967900 00000001:00080000:1.0:1787784735.883913:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967901 00000100:00100000:1.0:1787784735.884633:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:25908:x1874627658083456:12345-192.168.203.43@tcp:35:cat.0 Request processed in 1176us (1573us total) trans 4294967902 rc 0/0 00000100:00100000:1.0:1787784735.884646:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 188 00000020:00080000:1.0:1787784735.884652:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 3 transactions per RPC 00000100:00100000:2.0:1787784735.936250:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784735.936332:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658083584 00000100:00100000:1.0:1787784735.936342:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658083584 00002000:00100000:1.0:1787784735.936389:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (0->315) req@ffff98950895f840 x1874627658083584/t0(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:616/0 lens 808/0 e 0 to 0 dl 1787784746 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784735.936399:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98950895f840 with x1874627658083584 ext(0->315) 00002000:00010000:1.0:1787784735.936404:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98950895f840 00000100:00100000:1.0:1787784735.936437:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 89 00000100:00100000:1.0:1787784735.936458:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658083584:12345-192.168.203.43@tcp:4:cat.0 00000100:00100000:1.0:1787784735.936838:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658083584:12345-192.168.203.43@tcp:4:cat.0 Request processed in 380us (597us total) trans 4294967344 rc 0/0 00000100:00100000:1.0:1787784735.936846:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 89 00002000:00100000:1.0:1787784735.936853:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (0->315) req@ffff98950895f840 x1874627658083584/t4294967344(0) o4->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:616/0 lens 808/448 e 0 to 0 dl 1787784746 ref 1 fl Complete:/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784735.936861:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98950895f840 with x1874627658083584 ext(0->315) 00002000:00010000:1.0:1787784735.936864:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98950895f840 00000100:00100000:1.0:1787784735.959006:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784735.959511:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658083840 00000100:00100000:0.0:1787784735.959521:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658083840 00000100:00100000:0.0:1787784735.959579:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 933 00000100:00100000:0.0:1787784735.959598:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:25910:x1874627658083840:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784735.959620:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784735.959643:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5971 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d82 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.960334:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5978 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.960351:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5978 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.960363:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5978 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784735.960373:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 316 00010000:00010000:0.0:1787784735.960829:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5971 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d82 expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.960854:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5978 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d82 expref: 91 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784735.960863:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:0.0:1787784735.960866:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1787784735.960870:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030a00, rc 0) 00000100:00100000:0.0:1787784735.963460:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:25910:x1874627658083840:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3854us (4452us total) trans 4294967903 rc 0/0 00000100:00100000:0.0:1787784735.963480:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 933 00000100:00100000:0.0:1787784735.986265:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784735.986516:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658083968 00000100:00100000:3.0:1787784735.986527:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658083968 00000100:00100000:3.0:1787784735.986579:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 934 00000100:00100000:3.0:1787784735.986600:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:25910:x1874627658083968:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:3.0:1787784735.986920:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.986947:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.986960:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.986987:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.986993:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.987017:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.987023:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784735.987027:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.987035:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.987046:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:3.0:1787784735.987050:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784735.987054:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784735.987061:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5986 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784735.987136:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:25910:x1874627658083968:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 548us (881us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784735.987173:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 934 00000100:00100000:2.0:1787784736.024286:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784736.024395:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658084096 00000100:00100000:3.0:1787784736.024405:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658084096 00000100:00100000:3.0:1787784736.024465:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 935 00000100:00100000:3.0:1787784736.024475:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:25911:x1874627658084096:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784736.024510:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:2.0:1787784736.024620:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784736.026073:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc598d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d89 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784736.026195:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6a00/0xb0ce63137acc5994 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784736.026209:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6a00/0xb0ce63137acc5994 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784736.026221:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6a00/0xb0ce63137acc5994 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784736.026277:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 512 00000004:00200000:3.0:1787784736.026281:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784736.026304:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6a00/0xb0ce63137acc5994 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784736.026332:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc598d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d89 expref: 93 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784736.026351:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6a00/0xb0ce63137acc5994 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d89 expref: 92 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784736.026359:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff98951c407000) 00010000:00010000:3.0:1787784736.026361:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784736.026365:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6a00, rc 0) 00000100:00100000:3.0:1787784736.026431:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:25911:x1874627658084096:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1957us (2155us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784736.026439:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 935 00000100:00100000:1.0:1787784736.034892:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784736.035128:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658084480 00000100:00100000:3.0:1787784736.035138:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658084480 00000100:00100000:3.0:1787784736.035249:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:3.0:1787784736.035259:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:25911:x1874627658084480:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:3.0:1787784736.035488:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:25911:x1874627658084480:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 230us (605us total) trans 4294967904 rc 0/0 00000100:00100000:3.0:1787784736.035497:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 189 00000100:00100000:2.0:1787784736.045759:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784736.046593:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658084608 00000100:00100000:0.0:1787784736.046603:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658084608 00000100:00100000:0.0:1787784736.046670:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 40 00000100:00100000:0.0:1787784736.046679:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:25911:x1874627658084608:12345-192.168.203.43@tcp:103:lfs.0 00010000:00010000:0.0:1787784736.046693:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1787784736.046717:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5978 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d82 expref: 93 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784736.046737:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc58ec lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d3c expref: 92 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784736.046747:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff989505542d00) 00010000:00010000:0.0:1787784736.046752:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784736.046812:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:25911:x1874627658084608:12345-192.168.203.43@tcp:103:lfs.0 Request processed in 133us (1061us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784736.046819:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 40 00000100:00100000:3.0:1787784736.049626:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784736.051264:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658084736 00000100:00100000:2.0:1787784736.051278:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658084736 00000100:00100000:2.0:1787784736.051346:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 936 00000100:00100000:2.0:1787784736.051371:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:25911:x1874627658084736:12345-192.168.203.43@tcp:58:lfs.0 00000100:00100000:2.0:1787784736.051512:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:25911:x1874627658084736:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 143us (1894us total) trans 0 rc -1/-1 00000100:00100000:2.0:1787784736.051520:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 936 00000100:00100000:2.0:1787784736.241798:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784736.249100:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668385792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784736.249196:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784736.249306:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668385792 00000100:00100000:3.0:1787784736.249315:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668385792 00000100:00100000:3.0:1787784736.249383:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:3.0:1787784736.249392:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668385792:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787784736.249399:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00000024:3.0:1787784736.249416:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 3407872 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784736.249473:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668385792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784736.249481:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1787784736.249511:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668385792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784736.519059:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784736.519098:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784736.519136:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784736.519205:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784736.519233:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658084992 00000100:00100000:2.0:1787784736.519242:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658084992 00000100:00100000:2.0:1787784736.519293:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658085248 00000100:00100000:2.0:1787784736.519297:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658085248 00000100:00100000:2.0:1787784736.519322:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 294 00000100:00100000:2.0:1787784736.519332:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658084992:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784736.519397:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658084992:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 66us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784736.519405:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 294 00000100:00100000:2.0:1787784736.519422:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 295 00000100:00100000:2.0:1787784736.519428:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658085248:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784736.519454:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658085248:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784736.519478:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 295 00000100:00080000:2.0:1787784736.519505:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784736.519579:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658085120 00000100:00100000:3.0:1787784736.519593:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658085120 00000100:00100000:3.0:1787784736.519647:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 344 00000100:00100000:3.0:1787784736.519665:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658085120:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784736.519702:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658085120:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (604us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784736.519708:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 344 00000100:00080000:2.0:1787784736.520043:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784736.520291:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658084864 00000100:00100000:0.0:1787784736.520296:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658084864 00000100:00100000:0.0:1787784736.520338:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 541 00000100:00100000:0.0:1787784736.520345:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658084864:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784736.520386:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658084864:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (1338us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784736.520393:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 541 00010000:00010000:1.0:1787784736.689900:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784737.713355:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784738.290011:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784738.290048:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668385920:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784738.290139:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784738.290269:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668385920 00000100:00100000:1.0:1787784738.290278:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668385920 00000100:00100000:1.0:1787784738.290347:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:1.0:1787784738.290356:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668385920:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784738.290401:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 74776576 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784738.290449:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668385920:12345-0@lo:13:osp-pre-1-0.0 Request processed in 94us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784738.290457:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00080000:1.0:1787784738.290480:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784738.290531:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668385920:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784738.291377:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668386304:0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.291428:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784738.291472:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784738.291479:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ecf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668386176:0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.291539:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784738.291571:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386304 00000100:00100000:3.0:1787784738.291577:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386304 00000100:00100000:3.0:1787784738.291623:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386176 00000100:00100000:3.0:1787784738.291626:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386176 00000100:00100000:3.0:1787784738.291717:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:3.0:1787784738.291727:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668386304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.291778:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668386304:12345-0@lo:400:kworker.0 Request processed in 52us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784738.291785:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:3.0:1787784738.291800:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1787784738.291806:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668386176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.291836:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668386176:12345-0@lo:400:kworker.0 Request processed in 30us (297us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784738.291842:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00080000:3.0:1787784738.291870:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784738.291895:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ecf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668386176:0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.291905:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668386304:0@lo:400:kworker.0 00000100:00100000:2.0:1787784738.292230:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668386432:0@lo:400:kworker.0 00000100:00100000:2.0:1787784738.292278:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784738.292317:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1787784738.292322:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668386048:0@lo:400:kworker.0 00000100:00100000:2.0:1787784738.292352:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784738.292371:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668386560:0@lo:400:kworker.0 00000100:00100000:2.0:1787784738.292392:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784738.292808:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386432 00000100:00100000:3.0:1787784738.292814:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386432 00000100:00100000:3.0:1787784738.292871:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1787784738.292879:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668386432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.292997:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668386432:12345-0@lo:400:kworker.0 Request processed in 118us (717us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784738.293004:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1787784738.293029:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386048 00000100:00100000:3.0:1787784738.293045:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386048 00000100:00100000:3.0:1787784738.293071:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 542 00000100:00100000:3.0:1787784738.293077:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668386048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.293110:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668386048:12345-0@lo:400:kworker.0 Request processed in 33us (757us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784738.293116:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 542 00000100:00100000:3.0:1787784738.293144:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386560 00000100:00100000:3.0:1787784738.293178:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386560 00000100:00100000:3.0:1787784738.293205:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:3.0:1787784738.293211:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668386560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784738.293379:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668386560:12345-0@lo:400:kworker.0 Request processed in 168us (985us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784738.293387:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00080000:2.0:1787784738.293746:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784738.293752:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784738.293780:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668386048:0@lo:400:kworker.0 00000100:00100000:2.0:1787784738.293795:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668386560:0@lo:400:kworker.0 00000100:00100000:2.0:1787784738.293806:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668386432:0@lo:400:kworker.0 00010000:00010000:1.0:1787784738.738890:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784738.906799:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967903 is committed 00000001:00080000:0.0:1787784738.906837:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967897 is committed 00000001:00080000:0.0:1787784738.906845:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967898 is committed 00000001:00080000:0.0:1787784738.906854:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967904 is committed 00000001:00080000:2.0:1787784738.982953:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967344 is committed 00000001:00080000:2.0:1787784738.982991:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967343 is committed 00000001:00080000:2.0:1787784738.982997:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967342 is committed 00010000:00010000:1.0:1787784739.762208:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784740.800939:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784741.361396:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ecf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668386688:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784741.361463:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784741.361703:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386688 00000100:00100000:1.0:1787784741.361711:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386688 00000100:00100000:1.0:1787784741.361770:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:1.0:1787784741.361780:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668386688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784741.361813:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 76480512 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784741.361860:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668386688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (397us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784741.361867:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00080000:2.0:1787784741.362076:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784741.363421:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967342 gen 2 00000100:00100000:1.0:1787784741.363455:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ecf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668386688:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784741.433252:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.433996:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658085376 00000100:00100000:3.0:1787784741.434006:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658085376 00000100:00100000:3.0:1787784741.434081:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 937 00000100:00100000:3.0:1787784741.434092:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:25890:x1874627658085376:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1787784741.434221:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:25890:x1874627658085376:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 131us (977us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784741.434227:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 937 00000100:00080000:2.0:1787784741.434471:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784741.646977:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.647083:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.647117:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.647165:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.647186:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.647202:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.647223:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784741.647267:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658085760 00000100:00100000:3.0:1787784741.647275:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658085760 00000100:00100000:2.0:1787784741.648495:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658086272 00000100:00100000:2.0:1787784741.648503:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658086272 00000100:00100000:2.0:1787784741.648537:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658085888 00000100:00100000:2.0:1787784741.648540:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658085888 00000100:00100000:2.0:1787784741.648553:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658086144 00000100:00100000:2.0:1787784741.648555:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658086144 00000100:00100000:2.0:1787784741.648572:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 297 00000100:00100000:2.0:1787784741.648582:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658085760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784741.648653:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658085760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 72us (1569us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784741.648661:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 297 00000100:00100000:2.0:1787784741.648676:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 298 00000100:00100000:2.0:1787784741.648682:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658086272:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784741.648708:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658086272:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (1543us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784741.648713:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 298 00000100:00100000:2.0:1787784741.648724:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 299 00000100:00100000:2.0:1787784741.648729:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658085888:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784741.648750:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658085888:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (1547us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784741.648755:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 299 00000100:00100000:2.0:1787784741.648765:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:2.0:1787784741.648771:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658086144:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784741.648792:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658086144:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (1567us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784741.648797:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00080000:2.0:1787784741.648819:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784741.648823:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784741.650113:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658086016 00000100:00100000:1.0:1787784741.650124:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658086016 00000100:00100000:1.0:1787784741.650182:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658085632 00000100:00100000:1.0:1787784741.650186:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658085632 00000100:00100000:1.0:1787784741.650219:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 348 00000100:00100000:1.0:1787784741.650228:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2357:x1874627658086016:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784741.650285:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2357:x1874627658086016:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 59us (3168us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784741.650292:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 348 00000100:00100000:1.0:1787784741.654593:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:1.0:1787784741.654612:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:2357:x1874627658085632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784741.654699:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:2357:x1874627658085632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 93us (7511us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784741.654706:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00080000:1.0:1787784741.654742:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784741.655443:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658085504 00000100:00100000:0.0:1787784741.655452:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658085504 00000100:00100000:0.0:1787784741.655537:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 543 00000100:00100000:0.0:1787784741.655545:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658085504:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784741.655591:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658085504:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 47us (8624us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784741.655599:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 543 00010000:00010000:1.0:1787784741.809302:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784742.193418:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784742.193427:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784742.193716:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 24 count 2 00000040:00080000:3.0:1787784742.193736:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784742.836095:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787784743.261474:0:19279:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 18:52:21 (1787784741) 00000100:00100000:2.0:1787784743.409414:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784743.409437:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530506d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668386816:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784743.409512:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784743.409614:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386816 00000100:00100000:1.0:1787784743.409622:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386816 00000100:00100000:1.0:1787784743.409696:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:1.0:1787784743.409705:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668386816:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784743.409740:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 74776576 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784743.409784:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668386816:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784743.409792:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:1.0:1787784743.409837:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530506d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668386816:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784743.410335:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668387072:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.410371:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784743.410394:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1787784743.410400:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668386944:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.410435:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784743.410456:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668387456:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.410487:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784743.410512:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668387328:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.410536:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784743.410553:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784743.410558:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668387200:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.410578:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784743.410600:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668387072 00000100:00100000:0.0:1787784743.410617:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668387072 00000100:00100000:0.0:1787784743.410646:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668387456 00000100:00100000:0.0:1787784743.410649:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668387456 00000100:00100000:0.0:1787784743.410663:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668387200 00000100:00100000:0.0:1787784743.410666:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668387200 00000100:00100000:0.0:1787784743.410977:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:0.0:1787784743.410985:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668387072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411030:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668387072:12345-0@lo:400:kworker.0 Request processed in 45us (658us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.411037:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:0.0:1787784743.411055:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:0.0:1787784743.411061:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668387456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411085:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668387456:12345-0@lo:400:kworker.0 Request processed in 25us (597us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.411091:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00100000:0.0:1787784743.411102:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:0.0:1787784743.411108:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668387200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411135:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668387200:12345-0@lo:400:kworker.0 Request processed in 27us (556us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.411141:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:0.0:1787784743.411184:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668386944 00000100:00100000:0.0:1787784743.411188:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668386944 00000100:00100000:0.0:1787784743.411211:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 544 00000100:00100000:0.0:1787784743.411216:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668386944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411244:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668386944:12345-0@lo:400:kworker.0 Request processed in 27us (818us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.411250:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 544 00000100:00100000:0.0:1787784743.411289:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668387328 00000100:00100000:0.0:1787784743.411293:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668387328 00000100:00100000:0.0:1787784743.411320:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1787784743.411332:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668387328:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787784743.411364:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784743.411369:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784743.411372:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784743.411372:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668387328:12345-0@lo:400:kworker.0 Request processed in 31us (826us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.411378:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1787784743.411409:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668386944:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411421:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668387456:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411430:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668387072:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411457:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668387200:0@lo:400:kworker.0 00000100:00100000:0.0:1787784743.411536:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668387328:0@lo:400:kworker.0 00000100:00100000:2.0:1787784743.602007:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.604727:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658086400 00000100:00100000:0.0:1787784743.604742:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658086400 00000100:00100000:0.0:1787784743.604889:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 938 00000100:00100000:0.0:1787784743.604900:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:26514:x1874627658086400:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1787784743.605016:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+39:26514:x1874627658086400:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 117us (3017us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.605024:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 938 00000100:00080000:2.0:1787784743.609682:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784743.629973:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.630075:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658086528 00000100:00100000:0.0:1787784743.630086:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658086528 00000100:00100000:0.0:1787784743.630201:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 939 00000100:00100000:0.0:1787784743.630213:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:26515:x1874627658086528:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784743.630235:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784743.630279:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc599b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d90 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630356:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630372:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630386:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630400:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787784743.630558:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784743.630649:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630658:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630909:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630918:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630924:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630929:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.630933:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630951:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630964:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784743.630967:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.630971:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630978:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59a2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630990:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.630996:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631001:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631006:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.631008:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631013:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631020:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff989503a6da00) 00010000:00010000:0.0:1787784743.631023:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.631025:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631041:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59a9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631077:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc599b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d90 expref: 85 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631088:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784743.631090:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.631094:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc599b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d90 expref: 85 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.631103:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031000, rc 0) 00000100:00100000:0.0:1787784743.631240:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:26515:x1874627658086528:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1029us (1277us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784743.631248:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 939 00000100:00100000:3.0:1787784743.647583:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.647933:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658086784 00000100:00100000:0.0:1787784743.647943:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658086784 00000100:00100000:0.0:1787784743.647999:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 940 00000100:00100000:0.0:1787784743.648010:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:26515:x1874627658086784:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784743.648100:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784743.648126:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc57f0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ce8 expref: 84 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.648269:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:0.0:1787784743.648276:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784743.648684:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.648713:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 38 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.648722:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.648754:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 38 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.648768:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 38 pid: 5812 timeout: 634 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784743.648782:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668387584:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784743.648875:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32840 going to sleep for 11 seconds 00000100:00100000:2.0:1787784743.654841:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.654973:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668387584:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784743.654997:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784743.656497:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658086912 00000100:00100000:1.0:1787784743.656508:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658086912 00010000:00010000:1.0:1787784743.656552:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 39 pid: 5812 timeout: 634 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784743.656597:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:1.0:1787784743.656606:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2357:x1874627658086912:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784743.656620:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784743.656626:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 39 pid: 5812 timeout: 634 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.656639:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 39 pid: 5812 timeout: 634 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.656663:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc582f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0cfd expref: 39 pid: 5812 timeout: 634 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.656699:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:1.0:1787784743.656704:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.656713:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.656720:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.656752:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784743.656777:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.656785:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.656803:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.656810:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784743.656816:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2357:x1874627658086912:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 208us (1979us total) trans 0 rc 0/0 00010000:00010000:2.0:1787784743.656820:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784743.656834:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 286 00000004:00200000:2.0:1787784743.658212:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 512 00010000:00010000:2.0:1787784743.658226:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.658246:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.658253:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.658260:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784743.658265:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.658272:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.658284:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989503a6d900) 00010000:00010000:2.0:1787784743.658288:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784743.658292:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.658298:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc59b0 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784743.658316:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000726962d reply state 00000000e80f6180 transno 4294967905 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784743.658326:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784743.658392:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:26515:x1874627658086784:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 10383us (10816us total) trans 4294967905 rc 0/0 00000100:00100000:2.0:1787784743.658402:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 940 00010000:00010000:0.0:1787784743.663582:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.663599:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.663616:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.663622:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.663641:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.663649:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.663664:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff989530cb1c00) 00010000:00010000:0.0:1787784743.663667:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.663671:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.663678:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc59b7 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784743.676538:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.676639:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087040 00000100:00100000:1.0:1787784743.676648:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087040 00000100:00100000:1.0:1787784743.676707:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 941 00000100:00100000:1.0:1787784743.676717:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+84:26516:x1874627658087040:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784743.676742:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.676766:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc59be lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d97 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.676829:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc59c5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.676843:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc59c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.676870:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc59c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784743.676944:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784743.676949:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 25 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784743.676973:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc59c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.676997:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc59be lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d97 expref: 86 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.677014:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc59c5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d97 expref: 85 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.677022:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784743.677025:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784743.677028:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d200, rc 0) 00000100:00100000:1.0:1787784743.677299:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:26516:x1874627658087040:12345-192.168.203.43@tcp:101:bash.0 Request processed in 582us (768us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784743.677307:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 941 00000100:00100000:2.0:1787784743.684240:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.684338:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087168 00000100:00100000:1.0:1787784743.684347:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087168 00000100:00100000:1.0:1787784743.684403:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 942 00000100:00100000:1.0:1787784743.684414:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:26516:x1874627658087168:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784743.684435:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.684453:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc59cc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0d9e expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684526:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684540:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684550:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684565:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684573:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684580:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684683:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d400/0xb0ce63137acc59e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684692:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d400/0xb0ce63137acc59e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684699:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d400/0xb0ce63137acc59e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784743.684726:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 512 00000004:00200000:1.0:1787784743.684730:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784743.684750:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d400/0xb0ce63137acc59e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684774:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684780:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684786:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684792:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.684797:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684805:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684815:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784743.684818:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.684822:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684829:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc59d3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684841:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684848:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684854:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684865:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.684868:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684896:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684904:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff98951c407500) 00010000:00010000:1.0:1787784743.684906:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.684909:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684915:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc59da lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684941:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc59cc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d9e expref: 87 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684953:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d400/0xb0ce63137acc59e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d9e expref: 86 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.684960:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.684962:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784743.684964:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d400, rc 0) 00000100:00100000:1.0:1787784743.685038:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:26516:x1874627658087168:12345-192.168.203.43@tcp:101:bash.0 Request processed in 625us (806us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784743.685194:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 942 00000100:00100000:3.0:1787784743.688850:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.688938:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087296 00000100:00100000:0.0:1787784743.688947:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087296 00000100:00100000:0.0:1787784743.689000:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 943 00000100:00100000:0.0:1787784743.689011:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:26516:x1874627658087296:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784743.689030:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784743.689061:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc59e8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0da5 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689123:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689136:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689175:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689267:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689277:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689283:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689346:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689354:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689360:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689366:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.689369:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689376:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689386:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:0.0:1787784743.689389:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.689393:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689399:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc59ef lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689419:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689425:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689431:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689436:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.689439:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689445:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689453:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff989503a6da00) 00010000:00010000:0.0:1787784743.689456:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.689458:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689464:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc59f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689497:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc59e8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0da5 expref: 87 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689508:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:0.0:1787784743.689511:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.689521:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc59e8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0da5 expref: 87 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.689532:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030200, rc 0) 00000100:00100000:0.0:1787784743.689593:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:26516:x1874627658087296:12345-192.168.203.43@tcp:101:bash.0 Request processed in 584us (751us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784743.689601:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 943 00000100:00100000:3.0:1787784743.699609:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.701672:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087424 00000100:00100000:1.0:1787784743.701700:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087424 00000100:00100000:1.0:1787784743.701758:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 944 00000100:00100000:1.0:1787784743.701767:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:26517:x1874627658087424:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784743.701785:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.701805:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc59fd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0dac expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.701859:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.701889:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.701902:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784743.701957:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784743.701961:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 25 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784743.701980:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.702001:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc59fd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 40 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.702017:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.702026:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784743.702029:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784743.702031:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:1.0:1787784743.702317:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+39:26517:x1874627658087424:12345-192.168.203.43@tcp:101:rm.0 Request processed in 550us (2716us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784743.702325:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 944 00000100:00100000:2.0:1787784743.705640:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.705739:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087552 00000100:00100000:0.0:1787784743.705748:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087552 00000100:00100000:0.0:1787784743.705802:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 945 00000100:00100000:0.0:1787784743.705812:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:26517:x1874627658087552:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784743.705830:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784743.705850:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5a0b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0db3 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.705929:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.705943:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.705953:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.705965:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.705971:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.705977:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706091:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5a20 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706101:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5a20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706108:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5a20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784743.706136:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 512 00000004:00200000:0.0:1787784743.706139:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784743.706196:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5a20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706209:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706214:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706220:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706226:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.706229:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706237:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706248:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784743.706257:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.706263:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706270:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5a12 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706284:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706290:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706296:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706301:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.706304:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706311:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706575:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff989503a6da00) 00010000:00010000:0.0:1787784743.706579:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.706583:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706589:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5a19 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706623:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5a0b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0db3 expref: 41 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706649:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030a00/0xb0ce63137acc5a20 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0db3 expref: 40 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.706656:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:0.0:1787784743.706659:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784743.706661:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030a00, rc 0) 00000100:00100000:0.0:1787784743.706720:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26517:x1874627658087552:12345-192.168.203.43@tcp:101:rm.0 Request processed in 909us (1088us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.706727:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 945 00000100:00100000:0.0:1787784743.717427:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.717491:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087680 00000100:00100000:1.0:1787784743.717501:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087680 00000100:00100000:1.0:1787784743.717559:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 946 00000100:00100000:1.0:1787784743.717570:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26517:x1874627658087680:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784743.717590:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.717609:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a27 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0dba expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717678:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717693:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717703:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717718:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717725:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717733:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717793:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717813:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717819:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717825:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.717829:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717835:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717847:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.717851:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.717854:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717859:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a2e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717871:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717876:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717893:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717897:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.717907:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717913:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717933:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff98951c407500) 00010000:00010000:1.0:1787784743.717936:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.717939:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717945:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a35 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717978:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a27 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dba expref: 41 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.717987:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.717989:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.717993:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a27 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dba expref: 41 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.718003:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c800, rc 0) 00000100:00100000:1.0:1787784743.718123:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26517:x1874627658087680:12345-192.168.203.43@tcp:101:rm.0 Request processed in 553us (704us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784743.718131:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 946 00000100:00100000:3.0:1787784743.736923:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.737028:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087808 00000100:00100000:1.0:1787784743.737038:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087808 00000100:00100000:1.0:1787784743.737094:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 947 00000100:00100000:1.0:1787784743.737103:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26517:x1874627658087808:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784743.737137:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.737205:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a3c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0dc1 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737284:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737298:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737309:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737335:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737343:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737349:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737407:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737413:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737419:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737425:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.737436:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737443:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737455:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.737458:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.737462:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737469:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a43 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737481:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737487:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737493:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737498:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.737500:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737507:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737514:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff98951c407500) 00010000:00010000:1.0:1787784743.737516:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.737526:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737531:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737564:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a3c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dc1 expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737575:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.737577:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.737581:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a3c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dc1 expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.737590:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c800, rc 0) 00000100:00100000:1.0:1787784743.737652:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26517:x1874627658087808:12345-192.168.203.43@tcp:101:rm.0 Request processed in 549us (737us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784743.737660:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 947 00000100:00100000:3.0:1787784743.756142:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.757135:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658087936 00000100:00100000:1.0:1787784743.757165:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658087936 00000100:00100000:1.0:1787784743.757245:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 948 00000100:00100000:1.0:1787784743.757256:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26518:x1874627658087936:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784743.757276:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.757306:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a51 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0dc8 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757385:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757417:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757440:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757457:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757466:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757473:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757534:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757541:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757547:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757554:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.757558:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757566:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757577:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.757586:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.757592:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757599:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5a58 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757613:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757618:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757624:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757628:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.757630:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757636:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757643:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff98951c407500) 00010000:00010000:1.0:1787784743.757646:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.757648:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757654:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a5f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757686:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a51 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dc8 expref: 41 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757704:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.757707:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.757712:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a51 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dc8 expref: 41 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.757723:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c800, rc 0) 00000100:00100000:1.0:1787784743.757791:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26518:x1874627658087936:12345-192.168.203.43@tcp:101:cp.0 Request processed in 537us (1657us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784743.757799:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 948 00000100:00100000:3.0:1787784743.762899:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.763643:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658088064 00000100:00100000:1.0:1787784743.763654:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658088064 00000100:00100000:1.0:1787784743.763720:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 949 00000100:00100000:1.0:1787784743.763730:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26518:x1874627658088064:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784743.763751:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.763772:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5a66 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0dcf expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763837:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763850:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763861:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763875:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763905:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763912:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763981:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763988:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763993:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.763998:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.764002:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764008:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764018:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.764021:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.764026:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764032:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5a6d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764044:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764055:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764060:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764065:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.764068:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764073:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764080:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff98951c407900) 00010000:00010000:1.0:1787784743.764082:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.764086:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764091:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5a74 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764123:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5a66 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dcf expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.764133:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784743.764135:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.764139:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5a66 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dcf expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.765581:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d800, rc 0) 00000100:00100000:1.0:1787784743.765744:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26518:x1874627658088064:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2013us (2862us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784743.765753:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 949 00000100:00100000:3.0:1787784743.769348:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.769902:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658088320 00000100:00100000:1.0:1787784743.769912:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658088320 00000100:00100000:1.0:1787784743.769968:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 950 00000100:00100000:1.0:1787784743.769977:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:26518:x1874627658088320:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784743.769998:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.770016:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5a7b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0dd6 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.770133:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.770196:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.770208:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.770222:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.770230:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.770237:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787784743.771211:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x15:0x0], index 1: 0 00010000:00010000:1.0:1787784743.771438:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.771451:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.771460:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771676:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771687:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771693:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771699:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784743.771703:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771710:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771722:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:2.0:1787784743.771725:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784743.771730:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771736:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5a82 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784743.771750:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000575d7817 reply state 00000000a0ac455e transno 4294967906 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784743.771758:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784743.771803:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5a7b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 43 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771836:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 42 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.771844:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:2.0:1787784743.771846:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784743.771848:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d600, rc 0) 00000100:00100000:2.0:1787784743.771927:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26518:x1874627658088320:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1951us (2589us total) trans 4294967906 rc 0/0 00000100:00100000:2.0:1787784743.771936:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 950 00010000:00010000:1.0:1787784743.776803:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.776820:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.776828:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.776834:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784743.776839:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.776847:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.776864:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff98951c407500) 00010000:00010000:1.0:1787784743.776867:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784743.776872:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.776909:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5a89 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784743.780577:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.780863:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658088448 00000100:00100000:0.0:1787784743.780873:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658088448 00000100:00100000:0.0:1787784743.781098:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 244 00000100:00100000:0.0:1787784743.781109:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658088448:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784743.781127:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784743.781164:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ddd expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784743.781192:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x15:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784743.781210:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=534s v=(0 0) 00010000:00010000:0.0:1787784743.781214:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784743.781224:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.781242:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.781272:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030e00, rc 0) 00000100:00100000:0.0:1787784743.781385:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658088448:12345-192.168.203.43@tcp:101:cp.0 Request processed in 277us (816us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.781392:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 244 00000100:00080000:2.0:1787784743.784302:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784743.787976:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.788107:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658088576 00000100:00100000:1.0:1787784743.788117:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658088576 00000100:00100000:0.0:1787784743.788393:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:0.0:1787784743.788406:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:26518:x1874627658088576:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784743.788423:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784743.788441:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0de4 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.788541:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.788551:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784743.788556:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784743.788561:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.788598:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.788613:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 12 pid: 6671 timeout: 634 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784743.788647:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668387712:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784743.788773:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32840 going to sleep for 11 seconds 00000100:00100000:0.0:1787784743.791742:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784743.791909:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668387712:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784743.791942:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.791962:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031800, rc 0) 00000100:00100000:2.0:1787784743.792024:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:26518:x1874627658088576:12345-192.168.203.43@tcp:101:cp.0 Request processed in 3622us (4057us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784743.792033:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:3.0:1787784743.792115:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658088704 00000100:00100000:3.0:1787784743.792122:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658088704 00010000:00010000:3.0:1787784743.792188:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 12 pid: 6671 timeout: 634 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784743.792221:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 287 00000100:00100000:3.0:1787784743.792230:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658088704:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784743.792242:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784743.792263:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 12 pid: 6671 timeout: 634 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784743.792295:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 12 pid: 6671 timeout: 634 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784743.792319:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5a9e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ddd expref: 12 pid: 6671 timeout: 634 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784743.792333:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784743.792337:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784743.792344:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784743.792356:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784743.792388:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784743.792402:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668387840:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784743.792449:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503fa0480 going to sleep for 11 seconds 00000100:00100000:3.0:1787784743.796242:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668387840:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784743.796268:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784743.796320:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658088704:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 4090us (4585us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784743.796331:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 287 00000100:00100000:0.0:1787784743.797973:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.798099:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658088960 00000100:00100000:1.0:1787784743.798107:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658088960 00000100:00100000:1.0:1787784743.798195:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:1.0:1787784743.798204:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d7050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26518:x1874627658088960:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784743.798546:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d7050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26518:x1874627658088960:12345-192.168.203.43@tcp:35:cp.0 Request processed in 342us (579us total) trans 4294967907 rc 0/0 00000100:00100000:1.0:1787784743.798554:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 190 00000100:00100000:1.0:1787784743.848512:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.848685:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658089088 00000100:00100000:0.0:1787784743.848696:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658089088 00000100:00100000:0.0:1787784743.848913:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 951 00000100:00100000:0.0:1787784743.848925:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+86:26521:x1874627658089088:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784743.848944:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784743.848966:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5aac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0deb expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849029:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849042:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849052:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849066:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849073:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849079:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849425:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849435:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849443:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784743.849474:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00000004:00200000:0.0:1787784743.849478:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784743.849496:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849507:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849515:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849521:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849526:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849533:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.849536:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849543:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849571:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:0.0:1787784743.849574:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.849578:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849584:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ab3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849597:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849611:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849617:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849622:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784743.849625:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849631:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849639:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff989503a6df00) 00010000:00010000:0.0:1787784743.849641:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784743.849644:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849650:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5aba lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849694:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5aac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 88 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849719:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 87 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.849727:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:0.0:1787784743.849729:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784743.849732:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030200, rc 0) 00000100:00100000:0.0:1787784743.849809:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:26521:x1874627658089088:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 886us (1307us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784743.849817:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 951 00010000:00010000:0.0:1787784743.857460:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784743.857919:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784743.858005:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658089216 00000100:00100000:0.0:1787784743.858014:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658089216 00000100:00100000:0.0:1787784743.858090:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 246 00000100:00100000:0.0:1787784743.858110:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658089216:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784743.858129:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784743.858183:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5ac8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0df2 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784743.858243:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668387968:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784743.858318:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a329c0 going to sleep for 11 seconds 00000100:00080000:2.0:1787784743.858494:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:0.0:1787784743.865134:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1787784743.865143:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb mtime: 0 -> 1787784743 00002000:00010000:0.0:1787784743.865184:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb atime: 0 -> 1787784743 00002000:00010000:0.0:1787784743.865201:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb ctime: 0 -> 1787784743 00000100:00100000:0.0:1787784743.865224:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668387968:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784743.865259:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5ac8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df2 expref: 13 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.865285:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5ac8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df2 expref: 13 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784743.865299:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030e00, rc 0) 00000100:00100000:0.0:1787784743.865359:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658089216:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7260us (7446us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784743.865369:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 246 00000100:00100000:1.0:1787784743.875135:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784743.875278:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658089344 00000100:00100000:2.0:1787784743.875287:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658089344 00000100:00100000:2.0:1787784743.875348:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 952 00000100:00100000:2.0:1787784743.875358:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:26521:x1874627658089344:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784743.875379:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784743.875412:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5acf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0df9 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.875646:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.875664:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.875676:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784743.875734:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 512 00000004:00200000:2.0:1787784743.875738:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784743.875761:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.875846:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5acf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 89 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.875864:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 88 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784743.875941:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:2.0:1787784743.875945:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784743.875949:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784743.876060:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:26521:x1874627658089344:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 703us (933us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784743.876068:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 952 00000100:00100000:3.0:1787784743.887454:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.887667:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658089472 00000100:00100000:1.0:1787784743.887676:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658089472 00000100:00100000:1.0:1787784743.887739:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 247 00000100:00100000:1.0:1787784743.887750:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658089472:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784743.887769:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784743.887787:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5add lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e00 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784743.887845:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668388096:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784743.887948:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f840 going to sleep for 11 seconds 00000100:00100000:1.0:1787784743.891418:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668388096:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784743.891483:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5add lrc: 1/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e00 expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.891508:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5add lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e00 expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784743.891523:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784743.891585:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658089472:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3836us (4139us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784743.891594:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 247 00000100:00100000:2.0:1787784743.898277:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784743.898431:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658089728 00000100:00100000:1.0:1787784743.898441:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658089728 00000100:00100000:1.0:1787784743.898507:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 953 00000100:00100000:1.0:1787784743.898517:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:26521:x1874627658089728:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:1.0:1787784743.898788:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].24, 136 off11456 00000100:00100000:1.0:1787784743.898891:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:26521:x1874627658089728:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 375us (632us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784743.898900:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 953 00000001:00080000:1.0:1787784743.997327:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967907 is committed 00000001:00080000:1.0:1787784743.997343:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967905 is committed 00000040:00080000:3.0:1787784744.241851:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784744.241860:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784744.241922:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 25 count 3 00000040:00080000:3.0:1787784744.241928:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784744.242241:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bcb40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668388224:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784744.242569:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784744.246306:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bcb40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668388224:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784744.255349:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.255464:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658089856 00000100:00100000:2.0:1787784744.255474:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658089856 00000100:00100000:2.0:1787784744.255542:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 954 00000100:00100000:2.0:1787784744.255553:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:26531:x1874627658089856:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784744.255588:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784744.255697:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:26531:x1874627658089856:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 146us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.255705:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 954 00000100:00100000:0.0:1787784744.262824:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784744.262907:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658090112 00000100:00100000:1.0:1787784744.262918:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658090112 00000100:00100000:1.0:1787784744.263011:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 955 00000100:00100000:1.0:1787784744.263021:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:26531:x1874627658090112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784744.263050:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784744.263070:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc5ae4 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e07 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.263195:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5aeb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.263210:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5aeb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.263223:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5aeb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784744.263234:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:1.0:1787784744.263794:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc5ae4 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e07 expref: 44 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.263813:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5aeb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e07 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.263821:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:1.0:1787784744.263824:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784744.263827:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784744.263902:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26531:x1874627658090112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 882us (1086us total) trans 4294967908 rc 0/0 00000100:00100000:1.0:1787784744.263909:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 955 00000100:00100000:1.0:1787784744.268634:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784744.268682:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784744.268712:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658090240 00000100:00100000:1.0:1787784744.268735:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658090240 00000100:00100000:1.0:1787784744.268793:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 41 00000100:00100000:1.0:1787784744.268803:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2356:x1874627658090240:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784744.268827:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784744.268852:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5aeb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e07 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.268870:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:1.0:1787784744.268887:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784744.268940:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:2356:x1874627658090240:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 138us (257us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784744.268946:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 41 00000100:00100000:3.0:1787784744.269556:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658090368 00000100:00100000:3.0:1787784744.269563:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658090368 00000100:00100000:3.0:1787784744.269610:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 248 00000100:00100000:3.0:1787784744.269618:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658090368:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784744.269652:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.269670:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.269679:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784744.269684:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784744.269691:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.269719:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.269733:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 11 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784744.269757:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668388352:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784744.269826:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd10c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784744.279005:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668388352:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784744.279035:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784744.284960:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.285037:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658090496 00000100:00100000:2.0:1787784744.285047:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658090496 00002000:00100000:2.0:1787784744.285108:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x15:0x0] (0->157) req@ffff989530507c00 x1874627658090496/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:625/0 lens 648/0 e 0 to 0 dl 1787784755 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784744.285121:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530507c00 with x1874627658090496 ext(0->157) 00010000:00010000:2.0:1787784744.285131:0:18437:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 12 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.285182:0:18437:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 12 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784744.285192:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989530507c00 00000100:00100000:2.0:1787784744.285215:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 9 00000100:00100000:2.0:1787784744.285225:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658090496:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784744.285713:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658090496:12345-192.168.203.43@tcp:4:cp.0 Request processed in 487us (760us total) trans 4294967351 rc 0/0 00000100:00100000:2.0:1787784744.285722:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 9 00002000:00100000:2.0:1787784744.285730:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x15:0x0] (0->157) req@ffff989530507c00 x1874627658090496/t4294967351(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:625/0 lens 648/448 e 0 to 0 dl 1787784755 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784744.285745:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530507c00 with x1874627658090496 ext(0->157) 00010000:00010000:2.0:1787784744.285751:0:18437:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 14 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.285765:0:18437:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 14 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784744.285773:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989530507c00 00000100:00100000:1.0:1787784744.295238:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.295365:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658090624 00000100:00100000:2.0:1787784744.295385:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658090624 00010000:00010000:2.0:1787784744.295425:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 14 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.295465:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 288 00000100:00100000:2.0:1787784744.295475:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2355:x1874627658090624:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784744.295491:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784744.295514:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 14 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.295529:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 14 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.295553:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009af94d10/0xb0ce63137acc5aa5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0de4 expref: 14 pid: 13825 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.295591:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784744.295595:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784744.295602:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.295732:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.295763:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784744.295796:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.295819:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658090624:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 345us (590us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.295828:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 288 00010000:00010000:3.0:1787784744.295886:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.295913:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.295922:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.295931:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.295938:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784744.295943:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.295964:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.295978:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784744.295987:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc5af9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784744.296050:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658090368:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 26432us (27425us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784744.296058:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 248 00000100:00100000:1.0:1787784744.298872:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.298961:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658090880 00000100:00100000:2.0:1787784744.298971:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658090880 00000100:00100000:2.0:1787784744.299024:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 956 00000100:00100000:2.0:1787784744.299033:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26531:x1874627658090880:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784744.301639:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26531:x1874627658090880:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 2596us (2766us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.301653:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 956 00000100:00100000:3.0:1787784744.314955:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784744.315130:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658091008 00000100:00100000:1.0:1787784744.315141:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658091008 00000100:00100000:1.0:1787784744.315214:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 249 00000100:00100000:1.0:1787784744.315224:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658091008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784744.315241:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784744.315262:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e0e expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784744.315292:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x15:0x0] initial LVB size: 158, mtime: 0x6a8f6e27, atime: 0x0, ctime: 0x6a8f6e27, blocks: 0x1 00010000:00010000:1.0:1787784744.315314:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=535s v=(0 0) 00010000:00010000:1.0:1787784744.315319:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784744.315328:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 13 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.315362:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 13 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784744.315445:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784744.315505:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658091008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 283us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784744.315513:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 249 00000100:00100000:1.0:1787784744.318999:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784744.319518:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658091264 00000100:00100000:3.0:1787784744.319527:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658091264 00000100:00100000:3.0:1787784744.319587:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 957 00000100:00100000:3.0:1787784744.319598:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26531:x1874627658091264:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784744.319775:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26531:x1874627658091264:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 179us (786us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784744.319783:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 957 00000100:00100000:3.0:1787784744.332679:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.332891:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658091392 00000100:00100000:2.0:1787784744.332901:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658091392 00002000:00100000:2.0:1787784744.332949:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x15:0x0] (0->4095) req@ffff98951b5bcb40 x1874627658091392/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:625/0 lens 488/0 e 0 to 0 dl 1787784755 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784744.332960:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5bcb40 with x1874627658091392 ext(0->4095) 00002000:00010000:2.0:1787784744.332979:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98951b5bcb40 00000100:00100000:2.0:1787784744.333018:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 90 00000100:00100000:2.0:1787784744.333028:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2354:x1874627658091392:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784744.333409:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2354:x1874627658091392:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 381us (738us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784744.333417:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 90 00002000:00100000:2.0:1787784744.333425:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x15:0x0] (0->4095) req@ffff98951b5bcb40 x1874627658091392/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:625/0 lens 488/4536 e 0 to 0 dl 1787784755 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784744.333672:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5bcb40 with x1874627658091392 ext(0->4095) 00002000:00010000:2.0:1787784744.333677:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98951b5bcb40 00000100:00100000:3.0:1787784744.343299:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.343517:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658091520 00000100:00100000:2.0:1787784744.343527:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658091520 00000100:00100000:2.0:1787784744.343577:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 958 00000100:00100000:2.0:1787784744.343587:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26531:x1874627658091520:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784744.343617:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784744.343639:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5b07 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e15 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.343673:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.343687:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.343699:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.343992:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5b07 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 44 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.344012:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.344021:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:2.0:1787784744.344025:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784744.344029:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784744.344178:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26531:x1874627658091520:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 591us (888us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.344187:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 958 00000100:00100000:2.0:1787784744.360967:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784744.361599:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658091648 00000100:00100000:3.0:1787784744.361610:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658091648 00000100:00100000:3.0:1787784744.361676:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 959 00000100:00100000:3.0:1787784744.361687:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26531:x1874627658091648:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784744.361720:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784744.361835:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26531:x1874627658091648:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 149us (875us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784744.361844:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 959 00000100:00100000:1.0:1787784744.366793:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.367142:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658091776 00000100:00100000:2.0:1787784744.367194:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658091776 00000100:00100000:2.0:1787784744.367247:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 250 00000100:00100000:2.0:1787784744.367256:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658091776:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784744.367306:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367322:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784744.367326:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784744.367332:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367346:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367614:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367625:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367633:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367640:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784744.367644:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367654:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367668:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.367677:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b15 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.367755:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658091776:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 498us (971us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.367763:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 250 00000100:00100000:1.0:1787784744.374679:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.374759:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658092032 00000100:00100000:2.0:1787784744.374767:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658092032 00000100:00100000:2.0:1787784744.374817:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 960 00000100:00100000:2.0:1787784744.374826:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26531:x1874627658092032:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784744.374922:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.374940:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.374950:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.374986:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.374999:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 43 pid: 5811 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.375013:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668388480:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784744.375184:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7aea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784744.379887:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784744.381229:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658092160 00000100:00100000:0.0:1787784744.381240:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658092160 00010000:00010000:0.0:1787784744.381282:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 44 pid: 5811 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784744.381312:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 289 00000100:00100000:0.0:1787784744.381321:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658092160:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784744.381334:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784744.381341:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 44 pid: 5811 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.381355:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 44 pid: 5811 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.381401:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:0.0:1787784744.381406:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.381416:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.381424:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.381442:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784744.381493:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658092160:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 172us (1625us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784744.381500:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 289 00010000:00010000:2.0:1787784744.384296:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b0e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e15 expref: 43 pid: 5811 timeout: 635 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.384320:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668388480:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784744.384349:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384371:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384378:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384780:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384790:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384798:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384804:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784744.384808:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384816:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384832:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:2.0:1787784744.384835:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784744.384840:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.384846:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b1c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.385802:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26531:x1874627658092032:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 10973us (11195us total) trans 4294967909 rc 0/0 00000040:00080000:1.0:1787784744.385802:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784744.385808:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:2.0:1787784744.385820:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 960 00000040:00080000:1.0:1787784744.386331:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1787784744.386339:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784744.393829:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.394104:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658092416 00000100:00100000:2.0:1787784744.394115:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658092416 00000100:00100000:2.0:1787784744.394181:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 191 00000100:00100000:2.0:1787784744.394192:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26531:x1874627658092416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784744.395285:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:26531:x1874627658092416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1089us (1460us total) trans 4294967910 rc 0/0 00000100:00100000:2.0:1787784744.395297:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 191 00000040:00080000:2.0:1787784744.868121:0:19330:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784744.868136:0:19330:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784744.868216:0:19330:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 25 count 3 00000040:00080000:2.0:1787784744.868222:0:19330:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787784744.882697:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784744.941411:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784744.941485:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658092544 00000100:00100000:2.0:1787784744.941499:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658092544 00000100:00100000:2.0:1787784744.941567:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 251 00000100:00100000:2.0:1787784744.941579:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658092544:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784744.941602:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784744.941641:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e1c expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.941674:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784744.941679:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784744.941690:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.941712:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.941736:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784744.941817:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2355:x1874627658092544:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 240us (421us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.941826:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 251 00000100:00100000:0.0:1787784744.960166:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784744.960215:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658092800 00000100:00100000:0.0:1787784744.960227:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658092800 00000100:00100000:0.0:1787784744.960290:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 961 00000100:00100000:0.0:1787784744.960301:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:26532:x1874627658092800:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784744.960327:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784744.960352:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5b2a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e23 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.960480:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5b31 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.960498:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5b31 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.960522:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5b31 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784744.960539:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:0.0:1787784744.960694:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5b2a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e23 expref: 90 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.960711:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5b31 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e23 expref: 89 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.960722:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:0.0:1787784744.960725:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784744.960728:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030c00, rc 0) 00000100:00100000:0.0:1787784744.960796:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:26532:x1874627658092800:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 497us (662us total) trans 4294967911 rc 0/0 00000100:00100000:0.0:1787784744.960803:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 961 00000100:00100000:3.0:1787784744.965080:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784744.965141:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658092928 00000100:00100000:0.0:1787784744.965175:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658092928 00000100:00100000:0.0:1787784744.965252:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 252 00000100:00100000:0.0:1787784744.965268:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2355:x1874627658092928:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:0.0:1787784744.965328:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,1 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.965355:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.965383:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.965391:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784744.965397:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784744.965401:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 5/0,1 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.965445:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.965460:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 13 pid: 13825 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784744.965477:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668388608:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784744.965595:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.965605:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 12 pid: 6670 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784744.965614:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668388736:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784744.965638:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989510bc0b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784744.979776:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784744.980069:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784744.980143:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668388608:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784744.980239:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658093056 00000100:00100000:2.0:1787784744.980249:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658093056 00010000:00010000:2.0:1787784744.980297:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 14 pid: 13825 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.980339:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658093184 00000100:00100000:2.0:1787784744.980341:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658093184 00010000:00010000:2.0:1787784744.980355:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 13 pid: 6670 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784744.980376:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:2.0:1787784744.980386:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658093056:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784744.980408:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1787784744.980436:0:5796:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:2.0:1787784744.980443:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 14 pid: 13825 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980458:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 14 pid: 13825 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980518:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc5b23 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e1c expref: 14 pid: 13825 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980550:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784744.980621:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658093056:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 236us (860us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.980630:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:2.0:1787784744.980655:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:2.0:1787784744.980661:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658093184:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784744.980670:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784744.980679:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 13 pid: 6670 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980689:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 13 pid: 6670 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980706:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5b00 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e0e expref: 13 pid: 6670 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980717:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784744.980722:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784744.980728:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980752:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784744.980778:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784744.980797:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658093184:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 137us (728us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784744.980805:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:0.0:1787784744.983800:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668388736:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784744.983859:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.983993:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984003:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984049:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984059:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984068:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984077:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784744.984082:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984107:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984131:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.984139:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aabc3895/0xb0ce63137acc5b3f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784744.984279:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658092928:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 19008us (19203us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784744.984291:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 252 00000100:00100000:0.0:1787784744.989559:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784744.989604:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658093440 00000100:00100000:0.0:1787784744.989617:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658093440 00000100:00100000:0.0:1787784744.989681:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 192 00000100:00100000:0.0:1787784744.989694:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:26532:x1874627658093440:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:0.0:1787784744.989785:0:5814:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x19:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5b31 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e23 expref: 89 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784744.990283:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967912 00000001:00080000:0.0:1787784744.993075:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967913 00010000:00010000:0.0:1787784744.993362:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.993391:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 43 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.993410:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.993419:0:5814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.993470:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.993490:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 92 pid: 5813 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784744.993510:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668388864:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784744.993996:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 43 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784744.994008:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 43 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784744.994020:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668388992:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784744.994052:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643240 going to sleep for 11 seconds 00000100:00100000:3.0:1787784744.998297:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668388864:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784744.998319:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668388992:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784744.998354:0:5814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784744.999024:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784744.999572:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784745.000447:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658093568 00000100:00100000:0.0:1787784745.000519:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658093568 00010000:00010000:0.0:1787784745.000594:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 93 pid: 5813 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.000652:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658093696 00000100:00100000:0.0:1787784745.000656:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658093696 00010000:00010000:0.0:1787784745.000676:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 44 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.000700:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 292 00000100:00100000:0.0:1787784745.000710:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:2357:x1874627658093568:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784745.000728:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784745.000735:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 93 pid: 5813 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.000749:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 93 pid: 5813 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.000778:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030200/0xb0ce63137acc5ac1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0deb expref: 93 pid: 5813 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.000794:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:0.0:1787784745.000799:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.000808:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.000815:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784745.000850:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 293 00000100:00100000:2.0:1787784745.000858:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658093696:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784745.000880:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784745.000886:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 44 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.000896:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 44 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.000899:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:2357:x1874627658093568:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 190us (1880us total) trans 0 rc 0/0 00010000:00010000:2.0:1787784745.000911:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5a90 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dd6 expref: 44 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.000917:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 292 00010000:00010000:2.0:1787784745.000921:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:2.0:1787784745.000925:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.000933:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.000951:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.000978:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784745.000998:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2357:x1874627658093696:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 142us (1427us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784745.001004:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 293 00010000:00010000:2.0:1787784745.001063:0:5814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.001070:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784745.001263:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967914 00000001:00080000:2.0:1787784745.001515:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967915 00000001:00080000:2.0:1787784745.001577:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967916 00000100:00080000:1.0:1787784745.001757:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784745.005563:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895088512c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668389120:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1787784745.005673:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784745.005738:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668389120 00000100:00100000:2.0:1787784745.005746:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668389120 00000100:00100000:2.0:1787784745.005812:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 33 00000100:00100000:2.0:1787784745.005823:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668389120:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1787784745.005931:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668389120:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 109us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784745.005951:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 33 00000100:00080000:2.0:1787784745.005979:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784745.005998:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a329c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784745.006678:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895088512c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668389120:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1787784745.007358:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967917 00000100:00100000:2.0:1787784745.007448:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895088512c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668389248:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1787784745.007511:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784745.007551:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784745.007568:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668389248 00000100:00100000:0.0:1787784745.007575:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668389248 00000100:00100000:0.0:1787784745.007629:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:0.0:1787784745.007639:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668389248:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1787784745.008043:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5814:x1874627668389248:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 405us (532us total) trans 4294967352 rc 0/0 00000100:00100000:0.0:1787784745.008052:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1787784745.008343:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895088512c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668389248:0@lo:1000:mdt_rdpg00_000.0 00010000:00010000:2.0:1787784745.008372:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.008384:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.008393:0:5814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.008401:0:5814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.008406:0:5814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.008415:0:5814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.008433:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:2.0:1787784745.008437:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1787784745.008442:0:5814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.008450:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5b46 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784745.008860:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x3d:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1787784745.009216:0:5814:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].12, 64 off9032 00000004:00080000:2.0:1787784745.009230:0:5814:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 536096082369ns, next at 537096079587ns, rc = 0 00000001:00080000:2.0:1787784745.009239:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967918 00010000:00010000:2.0:1787784745.009290:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407700) 00010000:00010000:2.0:1787784745.009294:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1787784745.009303:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5b31 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e23 expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784745.009426:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:26532:x1874627658093440:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 19734us (19880us total) trans 4294967919 rc 0/0 00000100:00100000:2.0:1787784745.009437:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 192 00000020:00080000:2.0:1787784745.009443:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000001:00080000:2.0:1787784745.048843:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967351 is committed 00000100:00100000:1.0:1787784745.049740:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784745.053563:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658093824 00000100:00100000:2.0:1787784745.053583:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658093824 00000100:00100000:2.0:1787784745.053664:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 962 00000100:00100000:2.0:1787784745.053675:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:26533:x1874627658093824:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784745.053706:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784745.053732:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5b4d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e2a expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.053814:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.053832:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.053846:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054044:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054053:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054060:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054622:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b62 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054634:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b62 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054641:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b62 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784745.054687:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00000004:00200000:2.0:1787784745.054692:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784745.054714:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b62 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054727:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b62 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054736:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054744:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054761:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054768:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.054773:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054782:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054795:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:2.0:1787784745.054799:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784745.054804:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054811:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5b54 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054825:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054832:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054838:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054843:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.054846:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.054860:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.055134:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff989530cb1300) 00010000:00010000:2.0:1787784745.055136:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784745.055140:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.055173:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5b5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.055218:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5b4d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e2a expref: 97 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.055232:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b62 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e2a expref: 96 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.055240:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784745.055242:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784745.055245:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784745.055335:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:26533:x1874627658093824:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1661us (5603us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784745.055345:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 962 00000100:00100000:3.0:1787784745.073416:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.073527:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658094080 00000100:00100000:1.0:1787784745.073539:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658094080 00000100:00100000:1.0:1787784745.073622:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 42 00000100:00100000:1.0:1787784745.073632:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:26533:x1874627658094080:12345-192.168.203.43@tcp:103:lfs.0 00010000:00010000:1.0:1787784745.073653:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784745.073693:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5b62 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e2a expref: 96 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.073713:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:1.0:1787784745.073725:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784745.073790:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:26533:x1874627658094080:12345-192.168.203.43@tcp:103:lfs.0 Request processed in 160us (390us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784745.073798:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 42 00000100:00100000:1.0:1787784745.077123:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.077279:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658094208 00000100:00100000:3.0:1787784745.077291:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658094208 00000100:00100000:3.0:1787784745.077377:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 963 00000100:00100000:3.0:1787784745.077390:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:26533:x1874627658094208:12345-192.168.203.43@tcp:58:lfs.0 00010000:00010000:3.0:1787784745.077538:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.077560:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.077575:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784745.077832:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].25, 136 off11592 00000100:00100000:3.0:1787784745.078337:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:26533:x1874627658094208:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 945us (1220us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.078350:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 963 00000040:00080000:1.0:1787784745.393417:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784745.393432:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784745.393488:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 26 count 4 00000040:00080000:1.0:1787784745.393496:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784745.393649:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0a1c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668389376:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784745.393774:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527ccecc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784745.422488:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0a1c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668389376:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784745.425535:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784745.433579:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658094336 00000100:00100000:0.0:1787784745.433599:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658094336 00000100:00100000:0.0:1787784745.434280:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 964 00000100:00100000:0.0:1787784745.434295:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658094336:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784745.434348:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784745.434491:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658094336:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 198us (8970us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784745.434500:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 964 00000100:00100000:2.0:1787784745.443727:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.443821:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658094464 00000100:00100000:3.0:1787784745.443835:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658094464 00000100:00100000:3.0:1787784745.443971:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 965 00000100:00100000:3.0:1787784745.443984:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658094464:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1787784745.444194:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.444217:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.444234:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444661:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444696:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444704:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444739:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444748:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444755:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784745.444808:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00000004:00200000:2.0:1787784745.444813:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784745.444828:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444840:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444847:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444854:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444861:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.444874:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444889:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444904:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784745.444907:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784745.444912:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444919:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc5b70 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444935:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444941:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444946:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444952:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.444955:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444962:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444970:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x8876c877 (ffff989530cb1300) 00010000:00010000:2.0:1787784745.444972:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784745.444975:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.444991:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5b77 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.445010:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.445015:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.445021:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.445026:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.445029:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.445036:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.445044:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784745.445046:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784745.445049:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.445054:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5b7e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784745.445207:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658094464:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 1221us (1490us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784745.445218:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 965 00000100:00100000:2.0:1787784745.457121:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.457218:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658094720 00000100:00100000:3.0:1787784745.457231:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658094720 00000100:00100000:3.0:1787784745.457307:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 966 00000100:00100000:3.0:1787784745.457319:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658094720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784745.457346:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784745.457372:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5b85 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e31 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457507:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457525:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457554:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784745.457568:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:3.0:1787784745.457703:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457710:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457716:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457722:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784745.457726:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457735:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457749:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff989505542300) 00010000:00010000:3.0:1787784745.457752:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784745.457757:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457764:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5b8c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457813:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5b85 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e31 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457835:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784745.457838:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784745.457842:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5b85 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e31 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.457853:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784745.457949:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:26543:x1874627658094720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 621us (833us total) trans 4294967920 rc 301/301 00000100:00100000:3.0:1787784745.457958:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 966 00000100:00100000:0.0:1787784745.472470:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.473099:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658094848 00000100:00100000:3.0:1787784745.473112:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658094848 00000100:00100000:3.0:1787784745.473203:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 967 00000100:00100000:3.0:1787784745.473214:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:26543:x1874627658094848:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784745.473260:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784745.473724:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:26543:x1874627658094848:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 509us (1277us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.473734:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 967 00000100:00100000:2.0:1787784745.477620:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.477707:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658095104 00000100:00100000:3.0:1787784745.477719:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658095104 00000100:00100000:3.0:1787784745.477787:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 968 00000100:00100000:3.0:1787784745.477799:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:26543:x1874627658095104:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784745.477824:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784745.477851:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5b9a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e38 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.478861:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5ba1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.478893:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5ba1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.478907:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5ba1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784745.478920:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 0 00010000:00010000:3.0:1787784745.478982:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5b9a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e38 expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.479002:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5ba1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e38 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.479012:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784745.479015:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784745.479019:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784745.479103:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:26543:x1874627658095104:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1305us (1494us total) trans 4294967921 rc 0/0 00000100:00100000:3.0:1787784745.479113:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 968 00000100:00100000:2.0:1787784745.487280:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784745.487383:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658095360 00000100:00100000:0.0:1787784745.487396:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658095360 00000100:00100000:0.0:1787784745.487458:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 969 00000100:00100000:0.0:1787784745.487468:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:26543:x1874627658095360:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784745.487548:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784745.487583:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc5ba1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e38 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.487628:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542800) 00010000:00010000:0.0:1787784745.487641:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784745.487668:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.487684:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 94 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.487693:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 4/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.487730:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 94 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.487746:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 94 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.487762:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668389504:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784745.487882:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784745.490037:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668389504:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784745.490074:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784745.499314:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.499618:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658095488 00000100:00100000:1.0:1787784745.499631:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658095488 00010000:00010000:1.0:1787784745.499687:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 95 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.499731:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 294 00000100:00100000:1.0:1787784745.499742:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658095488:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784745.499761:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784745.499768:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 95 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.499785:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 95 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.499827:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5ad6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0df9 expref: 95 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.499853:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784745.499873:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.499884:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.499894:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.499927:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784745.499994:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:2356:x1874627658095488:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 254us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784745.500003:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 294 00010000:00010000:1.0:1787784745.500036:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500045:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500082:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500095:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500103:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500455:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500464:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500471:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500477:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784745.500482:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500490:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500500:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784745.500503:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784745.500507:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500513:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5baf lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500523:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500528:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500534:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500545:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784745.500548:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500554:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500562:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x25b5f604 (ffff98951c407400) 00010000:00010000:1.0:1787784745.500565:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784745.500567:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.500573:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5bb6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x25b5f604 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.500643:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:26543:x1874627658095360:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 13176us (13376us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784745.500651:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 969 00000100:00100000:0.0:1787784745.505483:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.505666:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658095616 00000100:00100000:3.0:1787784745.505679:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658095616 00000100:00100000:3.0:1787784745.505746:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 970 00000100:00100000:3.0:1787784745.505757:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:26543:x1874627658095616:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784745.505794:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784745.505824:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5bbd lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e3f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.506220:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.506239:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.506253:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.506369:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5bbd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.506390:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.506399:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:3.0:1787784745.506402:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784745.506406:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784745.506486:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:26543:x1874627658095616:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 730us (1017us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.506494:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 970 00000100:00100000:1.0:1787784745.509882:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784745.509966:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658095872 00000100:00100000:2.0:1787784745.509978:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658095872 00000100:00100000:2.0:1787784745.510047:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 971 00000100:00100000:2.0:1787784745.510058:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:26543:x1874627658095872:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784745.510195:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.510217:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.510232:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784745.510496:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001accfa9c reply state 00000000fb41f9e3 transno 4294967922 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784745.510507:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784745.510616:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:26543:x1874627658095872:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 559us (756us total) trans 4294967922 rc 0/0 00000100:00100000:2.0:1787784745.510624:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 971 00010000:00010000:2.0:1787784745.518813:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.518850:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.518859:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.518880:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.518886:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.518895:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.518915:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:2.0:1787784745.518918:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787784745.518924:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.518931:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bcb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.519376:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784745.519547:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658096000 00000100:00100000:0.0:1787784745.519559:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658096000 00000100:00100000:0.0:1787784745.519614:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 972 00000100:00100000:0.0:1787784745.519625:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:26543:x1874627658096000:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784745.519657:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784745.519680:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e46 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.519762:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.519776:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.519785:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:0.0:1787784745.519788:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784745.519791:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030e00, rc 0) 00000100:00100000:0.0:1787784745.519878:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:26543:x1874627658096000:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 254us (509us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784745.519886:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 972 00000100:00100000:0.0:1787784745.530502:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.530603:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658096128 00000100:00100000:1.0:1787784745.530616:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658096128 00000100:00100000:1.0:1787784745.530697:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:1.0:1787784745.530709:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:26543:x1874627658096128:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1787784745.530895:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:26543:x1874627658096128:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 188us (408us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784745.530902:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 193 00000100:00100000:1.0:1787784745.537752:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784745.537846:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658096384 00000100:00100000:2.0:1787784745.537858:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658096384 00000100:00100000:2.0:1787784745.537940:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:2.0:1787784745.537952:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:26543:x1874627658096384:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784745.538301:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26543:x1874627658096384:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 349us (561us total) trans 4294967923 rc 0/0 00000100:00100000:2.0:1787784745.538309:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 194 00000100:00100000:3.0:1787784745.548400:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784745.548592:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658096640 00000100:00100000:2.0:1787784745.548619:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658096640 00000100:00100000:3.0:1787784745.548782:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 973 00000100:00100000:3.0:1787784745.548795:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26543:x1874627658096640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784745.548951:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26543:x1874627658096640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 159us (563us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.548959:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 973 00000100:00100000:3.0:1787784745.552431:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784745.552818:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658096896 00000100:00100000:0.0:1787784745.552832:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658096896 00000100:00100000:0.0:1787784745.552915:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 974 00000100:00100000:0.0:1787784745.552927:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26543:x1874627658096896:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784745.552952:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784745.552979:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5bd9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e4d expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.553103:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.553120:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.553169:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.553215:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.553231:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 50 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.553247:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668389632:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784745.553355:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066436c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784745.557229:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.557314:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668389632:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784745.557352:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.559327:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658097024 00000100:00100000:1.0:1787784745.559343:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658097024 00010000:00010000:1.0:1787784745.559403:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 51 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.559461:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 295 00000100:00100000:1.0:1787784745.559473:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2357:x1874627658097024:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784745.559493:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784745.559502:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 51 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.559518:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 51 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.559546:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5bd2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e46 expref: 51 pid: 5812 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.559562:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:1.0:1787784745.559567:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.559579:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.559587:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.559615:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784745.559711:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2357:x1874627658097024:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 240us (2495us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784745.559720:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 295 00010000:00010000:1.0:1787784745.559878:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.559887:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784745.559908:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 0 00000004:00080000:1.0:1787784745.560343:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x13:0x0], index 0: 0 00010000:00010000:1.0:1787784745.560519:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560529:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560536:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560543:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784745.560549:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560557:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560570:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:1.0:1787784745.560574:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784745.560579:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560601:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5be0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560661:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5bd9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e4d expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560672:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784745.560675:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784745.560678:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5bd9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e4d expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.560687:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030c00, rc 0) 00000100:00100000:1.0:1787784745.560742:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26543:x1874627658096896:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 7818us (8326us total) trans 4294967924 rc 301/301 00000100:00100000:1.0:1787784745.560751:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 974 00000100:00100000:3.0:1787784745.566206:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.566314:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658097280 00000100:00100000:1.0:1787784745.566327:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658097280 00000100:00100000:1.0:1787784745.566384:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:1.0:1787784745.566394:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d7450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26543:x1874627658097280:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784745.566810:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d7450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:26543:x1874627658097280:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 415us (617us total) trans 4294967925 rc 0/0 00000100:00100000:1.0:1787784745.566819:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 195 00000100:00100000:1.0:1787784745.573620:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784745.574495:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658097408 00000100:00100000:2.0:1787784745.574512:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658097408 00000100:00100000:2.0:1787784745.574585:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 975 00000100:00100000:2.0:1787784745.574595:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:26543:x1874627658097408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784745.574625:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784745.574653:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bee lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e54 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.574743:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5bf5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.574759:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.574773:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784745.574816:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 0 00000004:00200000:2.0:1787784745.574821:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784745.574841:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.574884:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5bee lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e54 expref: 52 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.574904:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5bf5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e54 expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.574913:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:2.0:1787784745.574931:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1787784745.574936:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784745.575023:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:26543:x1874627658097408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 428us (1415us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784745.575031:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 975 00000100:00100000:1.0:1787784745.588041:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.588198:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658097536 00000100:00100000:3.0:1787784745.588212:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658097536 00000100:00100000:3.0:1787784745.588290:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 253 00000100:00100000:3.0:1787784745.588301:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658097536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784745.588326:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784745.588350:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e5b expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784745.588386:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x13:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1787784745.588411:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=536s v=(0 0) 00010000:00010000:3.0:1787784745.588416:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784745.588426:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.588458:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.588489:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784745.588557:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658097536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 258us (531us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.588564:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 253 00000100:00080000:3.0:1787784745.588586:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784745.591307:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.591412:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658097792 00000100:00100000:1.0:1787784745.591425:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658097792 00000100:00100000:1.0:1787784745.591526:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 976 00000100:00100000:1.0:1787784745.591537:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:26543:x1874627658097792:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784745.591684:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:26543:x1874627658097792:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 148us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784745.591691:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 976 00000100:00100000:3.0:1787784745.599228:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.599318:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658097920 00000100:00100000:1.0:1787784745.599329:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658097920 00000100:00100000:1.0:1787784745.599393:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 254 00000100:00100000:1.0:1787784745.599403:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:26543:x1874627658097920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784745.599425:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784745.599457:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e62 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.599496:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.599507:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784745.599512:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784745.599517:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 10 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.599556:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.599573:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 10 pid: 6669 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.599610:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5352c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668389760:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784745.599701:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce780 going to sleep for 11 seconds 00000100:00100000:1.0:1787784745.602355:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.602397:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658098048 00000100:00100000:1.0:1787784745.602405:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658098048 00010000:00010000:1.0:1787784745.602462:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 11 pid: 6669 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.602505:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00100000:1.0:1787784745.602516:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658098048:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784745.602533:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784745.602558:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 11 pid: 6669 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.602573:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 11 pid: 6669 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.602603:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784745.602608:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784745.602618:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.602629:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.602693:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.602709:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668389888:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784745.602779:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce900 going to sleep for 11 seconds 00010000:00010000:3.0:1787784745.603192:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc5bfc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e5b expref: 11 pid: 6669 timeout: 636 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784745.603211:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5352c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668389760:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784745.603238:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 10 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.603250:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c800, rc 0) 00000100:00100000:3.0:1787784745.603291:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:26543:x1874627658097920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3889us (4075us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.603299:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 254 00000100:00100000:2.0:1787784745.612642:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784745.612680:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658098176 00000100:00100000:2.0:1787784745.612690:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658098176 00002000:00100000:2.0:1787784745.612748:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x13:0x0] (0->157) req@ffff989530505680 x1874627658098176/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:626/0 lens 648/0 e 0 to 0 dl 1787784756 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784745.612760:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530505680 with x1874627658098176 ext(0->157) 00002000:00010000:2.0:1787784745.612766:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989530505680 00000100:00100000:2.0:1787784745.612796:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 91 00000100:00100000:2.0:1787784745.612806:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658098176:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1787784745.613451:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658098176:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 644us (820us total) trans 4294967345 rc 0/0 00000100:00100000:2.0:1787784745.613461:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 91 00002000:00100000:2.0:1787784745.613470:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x13:0x0] (0->157) req@ffff989530505680 x1874627658098176/t4294967345(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:626/0 lens 648/448 e 0 to 0 dl 1787784756 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784745.613479:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530505680 with x1874627658098176 ext(0->157) 00002000:00010000:2.0:1787784745.613482:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989530505680 00000100:00100000:2.0:1787784745.613627:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668389888:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784745.613652:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784745.613678:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658098048:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 11165us (11339us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784745.613697:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00100000:2.0:1787784745.617689:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.617771:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658098304 00000100:00100000:3.0:1787784745.617782:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658098304 00000100:00100000:3.0:1787784745.617847:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 977 00000100:00100000:3.0:1787784745.617857:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:26543:x1874627658098304:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:0.0:1787784745.683515:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967914 is committed 00000001:00080000:0.0:1787784745.683548:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967925 is committed 00000001:00080000:0.0:1787784745.683576:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967919 is committed 00000004:00080000:0.0:1787784745.683581:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 13 committed 00000020:00080000:0.0:1787784745.683606:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98953051d3c0 refcount 2 committed 0 result 0 batchid 15 00000020:00080000:0.0:1787784745.683612:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953051f200 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98953051dcc0 00000020:00080000:0.0:1787784745.683616:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953051f400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989401d16c00 00000020:00080000:0.0:1787784745.683659:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 15 committed batchid 14 00000020:00080000:0.0:1787784745.684015:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 14 00000100:00100000:0.0:1787784745.684049:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668390016:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787784745.684139:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784745.684196:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668390016 00000100:00100000:2.0:1787784745.684205:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668390016 00000100:00100000:2.0:1787784745.684269:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:2.0:1787784745.684279:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668390016:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1787784745.684313:0:10688:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:21 count 1 00002000:00010000:2.0:1787784745.684354:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x15:0x0] initial LVB size: 158, mtime: 0x6a8f6e27, atime: 0x0, ctime: 0x6a8f6e27, blocks: 0xb 00010000:00010000:2.0:1787784745.684365:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 3/0,1 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.684386:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=536s v=(0 0) 00010000:00010000:2.0:1787784745.684391:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784745.684399:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.684415:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.684425:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.684432:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784745.684563:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26543:x1874627658098304:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 66703us (66882us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.684585:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 977 00010000:00010000:2.0:1787784745.685237:0:10688:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787784745.685456:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784745.685465:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.685486:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784745.685491:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.685501:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.685520:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 1/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.685528:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc5c0a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.685548:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784745.685550:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784745.685552:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784745.687604:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3279:x1874627668390016:12345-0@lo:6:osp-syn-1-0.0 Request processed in 3315us (3453us total) trans 4294967353 rc 0/0 00000100:00100000:0.0:1787784745.687621:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000004:00080000:1.0:1787784745.687757:0:3279:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401fa4000/2, rc 0, transno 4294967353 00000100:00100000:1.0:1787784745.687775:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668390016:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787784745.688655:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.689215:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658098432 00000100:00100000:3.0:1787784745.689227:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658098432 00000100:00100000:3.0:1787784745.689297:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 256 00000100:00100000:3.0:1787784745.689307:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658098432:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:0.0:1787784745.748254:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967345 is committed 00000100:00100000:3.0:1787784745.748526:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658098432:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 59209us (59869us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784745.748541:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 256 00000100:00100000:0.0:1787784745.751503:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.751723:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658098688 00000100:00100000:1.0:1787784745.751737:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658098688 00000100:00100000:1.0:1787784745.751801:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 978 00000100:00100000:1.0:1787784745.751810:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26543:x1874627658098688:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784745.751998:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784745.752035:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5bf5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e54 expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.752060:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:1.0:1787784745.752067:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784745.752108:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.752125:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.752136:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784745.752430:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003dc98141 reply state 000000005d7c5876 transno 4294967926 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.752441:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784745.752554:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658098688:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 744us (1065us total) trans 4294967926 rc 0/0 00000100:00100000:1.0:1787784745.752562:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 978 00000100:00100000:0.0:1787784745.757275:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784745.757370:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.757385:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.757393:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.757399:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784745.757404:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.757413:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.757433:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:1.0:1787784745.757437:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784745.757442:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.757461:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c11 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784745.758317:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658098816 00000100:00100000:2.0:1787784745.758332:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658098816 00000100:00100000:2.0:1787784745.758405:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 979 00000100:00100000:2.0:1787784745.758418:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26543:x1874627658098816:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784745.758449:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784745.758472:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e69 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.758549:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.758564:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.758586:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:2.0:1787784745.758589:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784745.758592:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784745.758661:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658098816:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 246us (1400us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784745.758668:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 979 00000100:00100000:2.0:1787784745.762997:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.763193:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658098944 00000100:00100000:3.0:1787784745.763206:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658098944 00000100:00100000:3.0:1787784745.763271:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 257 00000100:00100000:3.0:1787784745.763283:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658098944:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1787784745.763622:0:6670:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1787784745.763627:0:6670:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb mtime from disk: 0 -> 1787784743 00002000:00010000:3.0:1787784745.763630:0:6670:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb atime from disk: 0 -> 1787784743 00002000:00010000:3.0:1787784745.763632:0:6670:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb ctime from disk: 0 -> 1787784743 00002000:00010000:3.0:1787784745.763635:0:6670:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1787784745.763726:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658098944:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 445us (744us total) trans 4294967346 rc 0/0 00000100:00100000:3.0:1787784745.763735:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 257 00000100:00100000:0.0:1787784745.766078:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.766296:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658099200 00000100:00100000:3.0:1787784745.766307:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658099200 00000100:00100000:3.0:1787784745.766371:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 980 00000100:00100000:3.0:1787784745.766382:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:26543:x1874627658099200:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1787784745.766463:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.766503:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.766514:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.766555:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.766569:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 43 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784745.766585:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668390144:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784745.766685:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784745.769357:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.769598:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668390144:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784745.769631:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.769648:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658099328 00000100:00100000:1.0:1787784745.769656:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658099328 00010000:00010000:1.0:1787784745.769705:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 44 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.769748:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 297 00000100:00100000:1.0:1787784745.769759:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2356:x1874627658099328:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784745.769777:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784745.769785:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 44 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.769797:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 44 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.769822:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc5bc4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e3f expref: 44 pid: 5811 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.769840:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:1.0:1787784745.769845:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.769855:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.769874:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.769909:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784745.769935:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.769941:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.769988:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.769991:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2356:x1874627658099328:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 233us (648us total) trans 0 rc 0/0 00010000:00010000:3.0:1787784745.769995:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.769999:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 297 00010000:00010000:3.0:1787784745.770017:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.770024:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784745.770029:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.770038:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.770054:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:3.0:1787784745.770057:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787784745.770061:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.770067:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c1f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784745.770109:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26543:x1874627658099200:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 3728us (4045us total) trans 0 rc -22/-22 00000100:00100000:3.0:1787784745.770117:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 980 00000100:00100000:2.0:1787784745.773260:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784745.773364:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658099456 00000100:00100000:0.0:1787784745.773376:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658099456 00000100:00100000:0.0:1787784745.773450:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 258 00000100:00100000:0.0:1787784745.773479:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658099456:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784745.773542:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.773568:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 10 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.773578:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784745.773583:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784745.773588:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.773827:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 10 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.773848:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 10 pid: 6670 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.773875:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668390272:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784745.773983:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989510bc06c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784745.776643:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668390272:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784745.776680:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784745.779309:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.779410:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658099584 00000100:00100000:1.0:1787784745.779422:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658099584 00010000:00010000:1.0:1787784745.779473:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 11 pid: 6670 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784745.779523:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 298 00000100:00100000:1.0:1787784745.779533:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658099584:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784745.779553:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784745.779581:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 11 pid: 6670 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.779596:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 11 pid: 6670 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.779624:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5c03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e62 expref: 11 pid: 6670 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.779644:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784745.779649:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784745.779656:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.779667:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.779728:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784745.779810:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658099584:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 277us (514us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784745.779819:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 298 00010000:00010000:0.0:1787784745.779906:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.779917:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.779958:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.779968:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.779977:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.779984:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784745.779988:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.779997:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.780009:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.780018:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c26 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.780074:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658099456:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 6598us (6826us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784745.780093:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 258 00000100:00100000:0.0:1787784745.783059:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784745.783142:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658099840 00000100:00100000:3.0:1787784745.783183:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658099840 00000100:00100000:3.0:1787784745.783252:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 981 00000100:00100000:3.0:1787784745.783263:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:26543:x1874627658099840:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784745.783361:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.783383:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.783416:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.783904:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.783919:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.783928:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.783966:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784745.783983:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 43 pid: 10850 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784745.784137:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668390400:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784745.784264:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d10c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784745.788768:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784745.788969:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658099968 00000100:00100000:0.0:1787784745.788981:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658099968 00010000:00010000:0.0:1787784745.789050:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 44 pid: 10850 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784745.789098:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 299 00000100:00100000:0.0:1787784745.789108:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658099968:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784745.789128:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784745.789135:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 44 pid: 10850 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.789195:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 44 pid: 10850 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.789224:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:0.0:1787784745.789230:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.789240:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.789248:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784745.789271:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784745.789334:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658099968:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 227us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784745.789360:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 299 00010000:00010000:2.0:1787784745.789661:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5c18 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e69 expref: 43 pid: 10850 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784745.789681:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668390400:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784745.789707:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.789717:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.789723:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784745.789992:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967927 00000001:00080000:2.0:1787784745.794098:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967928 00000001:00080000:2.0:1787784745.794269:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967929 00000100:00100000:2.0:1787784745.794396:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668390528:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787784745.794487:0:5813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784745.794536:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784745.794663:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668390528 00000100:00100000:0.0:1787784745.794672:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668390528 00000100:00100000:0.0:1787784745.794739:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 35 00000100:00100000:0.0:1787784745.794749:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668390528:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787784745.794843:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668390528:12345-0@lo:1000:mdt00_002.0 Request processed in 96us (356us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784745.794852:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 35 00000100:00100000:2.0:1787784745.795053:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668390528:0@lo:1000:mdt00_002.0 00000001:00080000:2.0:1787784745.795531:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967930 00000100:00100000:2.0:1787784745.795601:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668390656:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787784745.795642:0:5813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784745.795670:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063eb900 going to sleep for 11 seconds 00000100:00100000:1.0:1787784745.795873:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668390656 00000100:00100000:1.0:1787784745.795880:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668390656 00000100:00100000:1.0:1787784745.795921:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 36 00000100:00100000:1.0:1787784745.795930:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668390656:12345-0@lo:1000:mdt00_002.0 00000100:00080000:3.0:1787784745.795992:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784745.796163:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5813:x1874627668390656:12345-0@lo:1000:mdt00_002.0 Request processed in 216us (502us total) trans 4294967347 rc 0/0 00000100:00100000:1.0:1787784745.796171:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 36 00000100:00100000:2.0:1787784745.796343:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668390656:0@lo:1000:mdt00_002.0 00000001:00080000:2.0:1787784745.796443:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967931 00010000:00010000:2.0:1787784745.796478:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796490:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796499:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796506:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.796510:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796519:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796538:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff989505542e00) 00010000:00010000:2.0:1787784745.796540:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784745.796546:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796552:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc5c34 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796585:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796591:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796597:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796608:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.796612:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796618:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796626:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784745.796628:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784745.796631:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796638:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc5b69 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796660:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796673:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796682:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796689:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796694:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796700:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796709:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.796712:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796718:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796724:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784745.796726:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784745.796729:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796734:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5c3b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796743:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796749:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796755:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796760:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784745.796763:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796768:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796782:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff989505542300) 00010000:00010000:2.0:1787784745.796785:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784745.796788:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.796793:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc5c2d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784745.796986:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:26543:x1874627658099840:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 13722us (13939us total) trans 4294967932 rc 0/0 00000100:00100000:2.0:1787784745.796995:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 981 00000020:00080000:2.0:1787784745.796999:0:5813:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:3.0:1787784745.797308:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784745.797315:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784745.797782:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1787784745.797792:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784745.811728:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784745.811815:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658100224 00000100:00100000:1.0:1787784745.811828:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658100224 00000100:00100000:1.0:1787784745.811908:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 196 00000100:00100000:1.0:1787784745.811920:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:26543:x1874627658100224:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1787784745.812936:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1a:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1787784745.813137:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967933 00010000:00010000:1.0:1787784745.813235:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.813256:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.813270:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.813280:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.813296:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.813302:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784745.813327:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787784745.813452:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26543:x1874627658100224:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1531us (1736us total) trans 4294967934 rc 0/0 00000100:00100000:1.0:1787784745.813463:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 196 00000020:00080000:1.0:1787784745.813470:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1787784745.816383:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784745.816404:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.816417:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784745.816422:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.816433:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.816456:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff98951c407e00) 00010000:00010000:2.0:1787784745.816459:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787784745.816465:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.816471:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c42 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784745.816524:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784745.816526:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784745.816527:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784745.912195:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784746.028969:0:19382:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784746.028982:0:19382:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784746.031413:0:19382:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 26 count 4 00000040:00080000:2.0:1787784746.031428:0:19382:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784746.104118:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784746.107036:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658100480 00000100:00100000:1.0:1787784746.107059:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658100480 00000100:00100000:1.0:1787784746.107185:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 982 00000100:00100000:1.0:1787784746.107198:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+85:26546:x1874627658100480:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784746.107226:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784746.107255:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5c49 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e70 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.107357:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.107375:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.107391:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.107432:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.107440:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.107447:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.107852:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5c5e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109491:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5c5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109506:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5c5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784746.109528:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:1.0:1787784746.109685:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109693:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109701:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109707:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784746.109713:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109734:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109749:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff98951c407200) 00010000:00010000:1.0:1787784746.109753:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784746.109759:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109766:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5c50 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109779:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109786:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109792:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109798:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784746.109801:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109807:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109814:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff98951c407400) 00010000:00010000:1.0:1787784746.109817:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784746.109820:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109835:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5c57 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109920:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5c49 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e70 expref: 88 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109936:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d600/0xb0ce63137acc5c5e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e70 expref: 87 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784746.109944:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407e00) 00010000:00010000:1.0:1787784746.109946:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784746.109949:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d600, rc 0) 00000100:00100000:1.0:1787784746.111949:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:26546:x1874627658100480:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4735us (7827us total) trans 4294967935 rc 0/0 00000100:00100000:1.0:1787784746.111967:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 982 00000100:00100000:1.0:1787784746.127955:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784746.129823:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658100608 00000100:00100000:2.0:1787784746.129843:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658100608 00000100:00100000:2.0:1787784746.129955:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 983 00000100:00100000:2.0:1787784746.129967:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:26546:x1874627658100608:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:2.0:1787784746.130071:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130092:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130106:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130139:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130178:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130188:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130195:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784746.130218:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130227:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130243:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407e00) 00010000:00010000:2.0:1787784746.130247:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784746.130251:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.130258:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c6c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784746.130357:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:26546:x1874627658100608:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 393us (2417us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784746.130365:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 983 00000100:00100000:3.0:1787784746.132826:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784746.134246:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658100864 00000100:00100000:2.0:1787784746.134267:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658100864 00000100:00100000:2.0:1787784746.134350:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 197 00000100:00100000:2.0:1787784746.134362:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:26546:x1874627658100864:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:2.0:1787784746.134669:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:26546:x1874627658100864:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 308us (1855us total) trans 4294967936 rc 0/0 00000100:00100000:2.0:1787784746.134677:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 197 00000100:00100000:3.0:1787784746.187582:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784746.188363:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658100992 00000100:00100000:0.0:1787784746.188380:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658100992 00000100:00100000:0.0:1787784746.188480:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 259 00000100:00100000:0.0:1787784746.188492:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658100992:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:0.0:1787784746.188515:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784746.188548:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c73 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e77 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787784746.188694:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1787784746.188758:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x13:0x0] initial LVB size: 158, mtime: 0x6a8f6e27, atime: 0x6a8f6e27, ctime: 0x6a8f6e27, blocks: 0xb 00010000:00010000:0.0:1787784746.188804:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=537s v=(0 0) 00010000:00010000:0.0:1787784746.188810:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784746.188824:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c73 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e77 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784746.188853:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b584c56/0xb0ce63137acc5c73 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e77 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784746.188961:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030200, rc 0) 00000100:00100000:0.0:1787784746.189089:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2354:x1874627658100992:12345-192.168.203.43@tcp:101:diff.0 Request processed in 598us (1522us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784746.189098:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 259 00000100:00100000:1.0:1787784746.199003:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784746.199203:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658101248 00000100:00100000:2.0:1787784746.199216:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658101248 00000100:00100000:2.0:1787784746.199286:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 984 00000100:00100000:2.0:1787784746.199297:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:26551:x1874627658101248:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:2.0:1787784746.199331:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784746.199365:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c7a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e7e expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.199502:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5c81 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.199520:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5c81 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x19:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.199534:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5c81 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x19:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784746.199546:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:2.0:1787784746.199771:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5c7a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e7e expref: 91 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.199794:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fdc00/0xb0ce63137acc5c81 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x19:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e7e expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784746.199815:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff98951c407e00) 00010000:00010000:2.0:1787784746.199819:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1787784746.199822:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fdc00, rc 0) 00000100:00100000:2.0:1787784746.200187:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:26551:x1874627658101248:12345-192.168.203.43@tcp:101:diff.0 Request processed in 890us (1198us total) trans 4294967937 rc 0/0 00000100:00100000:2.0:1787784746.200197:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 984 00000100:00100000:0.0:1787784746.212021:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784746.212134:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658101376 00000100:00100000:2.0:1787784746.212173:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658101376 00002000:00100000:2.0:1787784746.212238:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x13:0x0] (0->4095) req@ffff989540738000 x1874627658101376/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:627/0 lens 488/0 e 0 to 0 dl 1787784757 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784746.212263:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540738000 with x1874627658101376 ext(0->4095) 00002000:00010000:2.0:1787784746.212271:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989540738000 00000100:00100000:2.0:1787784746.212313:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 92 00000100:00100000:2.0:1787784746.212325:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658101376:12345-192.168.203.43@tcp:3:diff.0 00000100:00100000:2.0:1787784746.212835:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658101376:12345-192.168.203.43@tcp:3:diff.0 Request processed in 509us (826us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784746.212846:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 92 00002000:00100000:2.0:1787784746.212869:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x13:0x0] (0->4095) req@ffff989540738000 x1874627658101376/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:627/0 lens 488/4536 e 0 to 0 dl 1787784757 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784746.212877:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540738000 with x1874627658101376 ext(0->4095) 00002000:00010000:2.0:1787784746.212881:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989540738000 00000100:00100000:1.0:1787784746.481396:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784746.481428:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668390784:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784746.481524:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784746.481656:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668390784 00000100:00100000:2.0:1787784746.481666:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668390784 00000100:00100000:2.0:1787784746.481742:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1787784746.481754:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668390784:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784746.481781:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 78184448 pending 0 free 3857711104 avail 3854565376 00000100:00100000:2.0:1787784746.481841:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668390784:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784746.481849:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1787784746.481927:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668390784:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784746.765266:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784746.765677:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784746.765722:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784746.765767:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784746.765785:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784746.765802:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784746.765835:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658101888 00000100:00100000:1.0:1787784746.765848:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658101888 00000100:00100000:0.0:1787784746.765868:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102144 00000100:00100000:0.0:1787784746.765875:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102144 00000100:00100000:0.0:1787784746.765939:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658101760 00000100:00100000:0.0:1787784746.765942:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658101760 00000100:00100000:3.0:1787784746.770802:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784746.771054:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658101632 00000100:00100000:2.0:1787784746.771066:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658101632 00000100:00100000:2.0:1787784746.771160:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102016 00000100:00100000:2.0:1787784746.771164:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102016 00000100:00100000:2.0:1787784746.771192:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 353 00000100:00100000:2.0:1787784746.771203:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:2357:x1874627658101632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784746.771320:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:2357:x1874627658101632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 119us (6069us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784746.771330:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 353 00000100:00100000:2.0:1787784746.771352:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 354 00000100:00100000:2.0:1787784746.771359:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658102016:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784746.771387:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658102016:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (5584us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784746.771393:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 354 00000100:00080000:1.0:1787784746.775176:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784746.785227:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658101504 00000100:00100000:3.0:1787784746.785248:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658101504 00000100:00100000:3.0:1787784746.785324:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 545 00000100:00100000:3.0:1787784746.785336:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658101504:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784746.785436:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658101504:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 102us (19667us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784746.785445:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 545 00000100:00100000:1.0:1787784746.792479:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102272 00000100:00100000:0.0:1787784746.795217:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 302 00000100:00100000:0.0:1787784746.795254:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658102144:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784746.795363:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658102144:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 128us (29639us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784746.795372:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 302 00000100:00100000:0.0:1787784746.795404:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 303 00000100:00100000:0.0:1787784746.795412:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658101888:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784746.795442:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658101888:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (29767us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784746.795448:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 303 00000100:00100000:0.0:1787784746.795463:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:0.0:1787784746.795471:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658101760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784746.795520:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658101760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 50us (29732us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784746.795526:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:1.0:1787784746.795829:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102272 00000100:00080000:1.0:1787784746.795886:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784746.795934:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:1.0:1787784746.795943:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658102272:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784746.796000:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658102272:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 58us (25213us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784746.796010:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 305 00010000:00010000:0.0:1787784746.933195:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784747.954036:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784748.529412:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784748.529444:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668390912:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784748.529544:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784748.529792:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668390912 00000100:00100000:1.0:1787784748.529802:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668390912 00000100:00100000:1.0:1787784748.529898:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:1.0:1787784748.529915:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668390912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784748.529975:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 76480512 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784748.530041:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668390912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 128us (497us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784748.530049:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:3.0:1787784748.530191:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668390912:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784748.530309:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668391168:0@lo:400:kworker.0 00000100:00100000:3.0:1787784748.530406:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784748.530442:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787784748.530449:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540666d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668391040:0@lo:400:kworker.0 00000100:00100000:3.0:1787784748.530494:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784748.530531:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668391552:0@lo:400:kworker.0 00000100:00100000:3.0:1787784748.530584:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784748.530626:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668391168 00000100:00100000:3.0:1787784748.530631:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668391168 00000100:00100000:3.0:1787784748.530680:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668391552 00000100:00100000:3.0:1787784748.530771:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668391552 00000100:00100000:3.0:1787784748.530800:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:3.0:1787784748.530807:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668391168:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1787784748.530881:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784748.530887:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784748.531099:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:2.0:1787784748.531114:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668391552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784748.531357:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668391168:12345-0@lo:400:kworker.0 Request processed in 548us (950us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784748.531369:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:3.0:1787784748.531413:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668391040 00000100:00100000:3.0:1787784748.531417:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668391040 00000100:00100000:3.0:1787784748.531456:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:3.0:1787784748.531464:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668391040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784748.535569:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668391552:12345-0@lo:400:kworker.0 Request processed in 4446us (4968us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784748.535589:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1787784748.535970:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668391040:12345-0@lo:400:kworker.0 Request processed in 4503us (5471us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784748.535982:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:3.0:1787784748.536060:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668391424:0@lo:400:kworker.0 00000100:00100000:3.0:1787784748.536129:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784748.536193:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784748.536201:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895406652c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668391296:0@lo:400:kworker.0 00000100:00100000:3.0:1787784748.536228:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784748.536261:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668391296 00000100:00100000:3.0:1787784748.536269:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668391296 00000100:00100000:3.0:1787784748.536329:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:3.0:1787784748.536339:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668391296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784748.536395:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668391296:12345-0@lo:400:kworker.0 Request processed in 58us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784748.536420:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00080000:3.0:1787784748.536441:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784748.536476:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895406652c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668391296:0@lo:400:kworker.0 00000100:00100000:1.0:1787784748.538028:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668391424 00000100:00100000:1.0:1787784748.538043:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668391424 00000100:00100000:1.0:1787784748.538120:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1787784748.538132:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668391424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784748.538252:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668391424:12345-0@lo:400:kworker.0 Request processed in 124us (2124us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784748.538260:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1787784748.538319:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668391424:0@lo:400:kworker.0 00000100:00080000:3.0:1787784748.538407:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784748.538524:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540666d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668391040:0@lo:400:kworker.0 00000100:00100000:1.0:1787784748.538644:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668391552:0@lo:400:kworker.0 00000100:00100000:1.0:1787784748.538660:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668391168:0@lo:400:kworker.0 00010000:00010000:1.0:1787784748.977504:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784750.001342:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784750.149509:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967352 is committed 00000001:00080000:1.0:1787784750.149528:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967353 is committed 00000100:00100000:3.0:1787784750.693461:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784750.693623:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102400 00000100:00100000:2.0:1787784750.693633:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102400 00000100:00100000:2.0:1787784750.693910:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 985 00000100:00100000:2.0:1787784750.693926:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26514:x1874627658102400:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1787784750.694036:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:26514:x1874627658102400:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 116us (585us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784750.694044:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 985 00000100:00080000:3.0:1787784750.694183:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784751.025283:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784751.130450:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967347 is committed 00000001:00080000:1.0:1787784751.130481:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967346 is committed 00000020:00080000:0.0:1787784751.158023:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 15 batchid updated 00000020:00080000:2.0:1787784751.158104:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 15 00000020:00080000:2.0:1787784751.158115:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 15 00000001:00080000:0.0:1787784751.161581:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967935 is committed 00000001:00080000:0.0:1787784751.161600:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967934 is committed 00000001:00080000:0.0:1787784751.161700:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967937 is committed 00000020:00080000:0.0:1787784751.161719:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989506643d80 refcount 2 committed 0 result 0 batchid 16 00000020:00080000:0.0:1787784751.161725:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895305ce900 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989506643180 00000020:00080000:0.0:1787784751.161728:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989508854100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895393c3400 00000020:00080000:2.0:1787784751.161762:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 16 committed batchid 15 00000020:00080000:2.0:1787784751.162112:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 15 00000100:00100000:1.0:1787784751.602098:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011eda40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668391680:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784751.602216:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784751.602426:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668391680 00000100:00100000:0.0:1787784751.602433:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668391680 00000100:00100000:0.0:1787784751.602517:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:0.0:1787784751.602527:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668391680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784751.602572:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 78184448 pending 0 free 3857711104 avail 3855613952 00000100:00080000:3.0:1787784751.602868:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784751.602908:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668391680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 381us (692us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784751.602916:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:1.0:1787784751.603067:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967347 gen 2 00000004:00080000:1.0:1787784751.603073:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967347 committed transno 4294967347 00000100:00100000:1.0:1787784751.603101:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011eda40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668391680:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784751.886754:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784751.886803:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784751.886912:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102528 00000100:00100000:0.0:1787784751.886912:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784751.886922:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102528 00000100:00100000:0.0:1787784751.886962:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784751.886969:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 547 00000100:00100000:1.0:1787784751.886983:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658102528:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784751.887052:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658102528:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 70us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784751.887060:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 547 00000100:00100000:1.0:1787784751.887220:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658103040 00000100:00100000:1.0:1787784751.887224:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658103040 00000100:00100000:1.0:1787784751.887263:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 358 00000100:00100000:1.0:1787784751.887270:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2357:x1874627658103040:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784751.887307:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2357:x1874627658103040:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (395us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784751.887314:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 358 00000100:00100000:0.0:1787784751.887644:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784751.887666:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784751.887681:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102784 00000100:00100000:2.0:1787784751.887685:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102784 00000100:00100000:1.0:1787784751.887713:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658103296 00000100:00100000:1.0:1787784751.887717:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658103296 00000100:00100000:2.0:1787784751.887724:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 307 00000100:00100000:2.0:1787784751.887731:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658102784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784751.887741:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 308 00000100:00100000:1.0:1787784751.887747:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658103296:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784751.887764:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658102784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (961us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784751.887768:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658103296:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (806us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784751.887771:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 307 00000100:00100000:1.0:1787784751.887782:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 308 00000100:00080000:2.0:1787784751.887788:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784751.887792:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784751.887904:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784751.888206:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102656 00000100:00100000:0.0:1787784751.888210:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102656 00000100:00100000:0.0:1787784751.888261:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 359 00000100:00100000:0.0:1787784751.888278:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:2357:x1874627658102656:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784751.888315:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:2357:x1874627658102656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 47us (672us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784751.888322:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 359 00000100:00080000:0.0:1787784751.888347:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784751.888704:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658102912 00000100:00100000:2.0:1787784751.888710:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658102912 00000100:00100000:2.0:1787784751.888737:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658103168 00000100:00100000:2.0:1787784751.888740:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658103168 00000100:00100000:2.0:1787784751.888753:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 309 00000100:00100000:2.0:1787784751.888759:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658102912:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784751.888788:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658102912:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (1121us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784751.888794:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 309 00000100:00100000:2.0:1787784751.888803:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00100000:2.0:1787784751.888809:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658103168:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784751.888840:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658103168:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (936us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784751.888846:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00080000:1.0:1787784751.891387:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784751.891395:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787784752.052310:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787784752.587067:0:19584:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 18:52:30 (1787784750) 00000100:00100000:0.0:1787784752.867889:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784752.868350:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658103424 00000100:00100000:3.0:1787784752.868365:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658103424 00000100:00100000:3.0:1787784752.868465:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 986 00000100:00100000:3.0:1787784752.868478:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:27154:x1874627658103424:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00080000:1.0:1787784752.870297:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784752.871210:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:27154:x1874627658103424:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 2715us (3317us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784752.871237:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 986 00000100:00100000:1.0:1787784752.895477:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784752.896278:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658103552 00000100:00100000:0.0:1787784752.896291:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658103552 00000100:00100000:0.0:1787784752.896375:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 987 00000100:00100000:0.0:1787784752.896386:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:27155:x1874627658103552:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784752.896413:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784752.896439:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5c8f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e85 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.896522:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.896551:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.896565:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.899347:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.899370:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.899383:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909701:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909719:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909727:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909733:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784752.909737:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909744:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909760:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:2.0:1787784752.909763:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.909767:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909773:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5c96 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909797:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909804:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909810:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909827:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784752.909830:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909836:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909844:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff989530cb1c00) 00010000:00010000:2.0:1787784752.909847:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.909850:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909856:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5c9d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909901:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5c8f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e85 expref: 88 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909911:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:2.0:1787784752.909913:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.909924:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5c8f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e85 expref: 88 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.909934:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:2.0:1787784752.909999:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:27155:x1874627658103552:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 13613us (14537us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784752.910008:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 987 00000100:00100000:1.0:1787784752.920268:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784752.920406:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658103808 00000100:00100000:0.0:1787784752.920417:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658103808 00000100:00100000:0.0:1787784752.920477:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 988 00000100:00100000:0.0:1787784752.920488:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:27155:x1874627658103808:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784752.920554:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784752.920581:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc59c5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0d97 expref: 87 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.920600:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784752.920605:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784752.921073:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.921089:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.921098:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.921130:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784752.921263:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 39 pid: 5812 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784752.921281:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073ad00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668391808:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784752.921424:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784752.930761:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784752.930888:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073ad00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668391808:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784752.930919:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784752.933074:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658103936 00000100:00100000:2.0:1787784752.933086:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658103936 00010000:00010000:2.0:1787784752.933131:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 40 pid: 5812 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784752.933322:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:2.0:1787784752.933333:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2357:x1874627658103936:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784752.933348:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784752.933355:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 40 pid: 5812 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.933369:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 40 pid: 5812 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.933391:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5a04 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0dac expref: 40 pid: 5812 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.933406:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:2.0:1787784752.933412:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.933451:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.933459:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.933497:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784752.933566:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+39:2357:x1874627658103936:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 233us (2813us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784752.933574:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 300 00010000:00010000:1.0:1787784752.934176:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.934186:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.934212:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.934223:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.934232:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784752.936108:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 512 00010000:00010000:1.0:1787784752.936124:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.936134:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.936322:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.936330:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784752.936334:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.936343:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.936357:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784752.936360:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784752.936364:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.936371:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ca4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784752.936392:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000da51e5a6 reply state 000000008e21d45f transno 4294967938 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784752.936400:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784752.936773:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:27155:x1874627658103808:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 16282us (16509us total) trans 4294967938 rc 0/0 00000100:00100000:1.0:1787784752.936783:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 988 00010000:00010000:2.0:1787784752.941343:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.941361:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.941368:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.941375:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784752.941380:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.941387:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.941405:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff98951c407400) 00010000:00010000:2.0:1787784752.941408:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.941423:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.941451:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cab lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784752.945589:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784752.945599:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784752.946081:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 26 count 3 00000040:00080000:3.0:1787784752.946091:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784752.955720:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784752.956002:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104064 00000100:00100000:1.0:1787784752.956012:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104064 00000100:00100000:1.0:1787784752.956075:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 989 00000100:00100000:1.0:1787784752.956085:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+87:27156:x1874627658104064:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784752.956124:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784752.956260:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5cb2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e8c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.956331:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5cb9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.956346:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5cb9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.956357:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5cb9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784752.956419:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784752.956424:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 26 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784752.956447:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5cb9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.956474:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5cb2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e8c expref: 89 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.956492:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5cb9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e8c expref: 88 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.956500:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784752.956503:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784752.956506:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c600, rc 0) 00000100:00100000:1.0:1787784752.956566:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:27156:x1874627658104064:12345-192.168.203.43@tcp:101:bash.0 Request processed in 483us (856us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784752.956574:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 989 00000100:00100000:0.0:1787784752.961565:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784752.961661:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104192 00000100:00100000:2.0:1787784752.961681:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104192 00000100:00100000:2.0:1787784752.961738:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 990 00000100:00100000:2.0:1787784752.961747:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+88:27156:x1874627658104192:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784752.961766:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784752.961785:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5cc0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e93 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.961853:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.961868:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.961878:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.961892:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.961899:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.961905:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962002:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5cd5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962023:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5cd5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962031:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5cd5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784752.962064:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 512 00000004:00200000:2.0:1787784752.962068:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784752.962088:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5cd5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962100:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962107:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962114:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962119:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784752.962123:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962130:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962141:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:2.0:1787784752.962144:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.962560:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962578:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5cc7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962591:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962596:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962601:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962606:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784752.962609:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962616:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962624:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff989530cb1c00) 00010000:00010000:2.0:1787784752.962627:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.962630:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962635:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5cce lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962664:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5cc0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e93 expref: 90 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962693:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc800/0xb0ce63137acc5cd5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e93 expref: 89 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.962707:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784752.962709:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784752.962712:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc800, rc 0) 00000100:00100000:2.0:1787784752.962835:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:27156:x1874627658104192:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1087us (1278us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784752.962843:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 990 00000100:00100000:3.0:1787784752.968274:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784752.968518:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104320 00000100:00100000:1.0:1787784752.968528:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104320 00000100:00100000:1.0:1787784752.968580:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 991 00000100:00100000:1.0:1787784752.968589:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:27156:x1874627658104320:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784752.968610:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784752.968632:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5cdc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0e9a expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968701:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968715:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968727:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968743:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968750:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968756:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968840:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968848:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968870:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968875:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784752.968879:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968887:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968898:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784752.968902:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784752.968906:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968912:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5ce3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968924:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968930:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968936:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968941:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784752.968944:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968960:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968967:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff98951c407900) 00010000:00010000:1.0:1787784752.968969:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784752.968972:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.968978:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cea lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.969009:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5cdc lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e9a expref: 90 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.969019:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784752.969021:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784752.969025:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5cdc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e9a expref: 90 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.969033:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c800, rc 0) 00000100:00100000:1.0:1787784752.969092:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:27156:x1874627658104320:12345-192.168.203.43@tcp:101:bash.0 Request processed in 505us (828us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784752.969099:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 991 00000100:00100000:3.0:1787784752.986821:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784752.986921:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104448 00000100:00100000:1.0:1787784752.986931:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104448 00000100:00100000:1.0:1787784752.987081:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 992 00000100:00100000:1.0:1787784752.987092:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+39:27157:x1874627658104448:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784752.987126:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784752.987182:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5cf1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ea1 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.987245:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.987259:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.987271:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784752.987332:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784752.987336:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 26 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784752.987362:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.987385:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c800/0xb0ce63137acc5cf1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.987417:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 40 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.987426:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784752.987429:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784752.987432:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:1.0:1787784752.987497:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+40:27157:x1874627658104448:12345-192.168.203.43@tcp:101:rm.0 Request processed in 407us (698us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784752.987506:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 992 00000100:00100000:0.0:1787784752.990723:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784752.990979:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104576 00000100:00100000:1.0:1787784752.990989:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104576 00000100:00100000:1.0:1787784752.991041:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 993 00000100:00100000:1.0:1787784752.991050:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+40:27157:x1874627658104576:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784752.991067:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784752.991085:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ea8 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991179:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991193:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991205:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991220:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991227:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991233:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991328:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc5d14 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991336:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc5d14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991358:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc5d14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784752.991387:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 512 00000004:00200000:1.0:1787784752.991390:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784752.991410:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc5d14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991436:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991443:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991450:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991456:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784752.991460:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991467:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991478:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784752.991481:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784752.991485:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991491:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d06 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991514:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991520:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991527:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991531:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784752.991534:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991544:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991551:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff98951c407900) 00010000:00010000:1.0:1787784752.991553:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784752.991556:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991561:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d0d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991587:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5cff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea8 expref: 42 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991600:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1de00/0xb0ce63137acc5d14 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea8 expref: 41 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784752.991618:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784752.991621:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784752.991623:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1de00, rc 0) 00000100:00100000:1.0:1787784752.991685:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27157:x1874627658104576:12345-192.168.203.43@tcp:101:rm.0 Request processed in 636us (970us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784752.991693:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 993 00000100:00100000:3.0:1787784752.993992:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784752.994416:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104704 00000100:00100000:2.0:1787784752.994426:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104704 00000100:00100000:2.0:1787784752.994516:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 994 00000100:00100000:2.0:1787784752.994527:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27157:x1874627658104704:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784752.994546:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784752.994581:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5d1b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0eaf expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994689:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994703:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994713:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994729:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994736:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994752:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994825:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994832:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994838:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994843:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784752.994847:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994854:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994865:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784752.994868:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.994872:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994879:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d22 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994892:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994898:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994911:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994917:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784752.994920:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994926:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994934:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff989530cb1c00) 00010000:00010000:2.0:1787784752.994937:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.994940:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994945:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d29 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994978:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5d1b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eaf expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.994987:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784752.994990:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784752.994993:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5d1b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eaf expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784752.995003:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784752.995066:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27157:x1874627658104704:12345-192.168.203.43@tcp:101:rm.0 Request processed in 541us (1082us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784752.995082:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 994 00000100:00100000:3.0:1787784753.003000:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784753.003216:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104832 00000100:00100000:1.0:1787784753.003232:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104832 00000100:00100000:1.0:1787784753.003307:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 995 00000100:00100000:1.0:1787784753.003318:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27157:x1874627658104832:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784753.003339:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784753.003360:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d30 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0eb6 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003425:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003439:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003450:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003465:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003473:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003498:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003562:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003569:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003575:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003581:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784753.003585:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003593:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003604:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784753.003606:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.003610:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003615:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d37 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003627:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003633:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003645:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003650:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784753.003653:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003659:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003667:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff98951c407900) 00010000:00010000:1.0:1787784753.003686:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.003690:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003696:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d3e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003728:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d30 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eb6 expref: 42 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003739:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784753.003741:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.003745:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d30 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eb6 expref: 42 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.003753:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784753.003827:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27157:x1874627658104832:12345-192.168.203.43@tcp:101:rm.0 Request processed in 511us (851us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784753.003843:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 995 00000100:00100000:0.0:1787784753.028042:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784753.028243:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658104960 00000100:00100000:1.0:1787784753.028258:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658104960 00000100:00100000:1.0:1787784753.028335:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 996 00000100:00100000:1.0:1787784753.028345:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27158:x1874627658104960:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784753.028366:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784753.028387:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d45 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ebd expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028478:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028493:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028505:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028520:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028527:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028534:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028602:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028625:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028631:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028637:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784753.028641:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028647:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028661:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784753.028664:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.028668:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028682:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d4c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028694:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028699:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028705:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028709:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784753.028715:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028721:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028727:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff98951c407900) 00010000:00010000:1.0:1787784753.028729:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.028975:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.028983:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.029017:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d45 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ebd expref: 42 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.029027:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784753.029029:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.029032:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d45 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ebd expref: 42 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.029042:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784753.029113:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27158:x1874627658104960:12345-192.168.203.43@tcp:101:cp.0 Request processed in 770us (1088us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784753.029120:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 996 00000100:00100000:3.0:1787784753.035497:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784753.035653:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658105088 00000100:00100000:1.0:1787784753.035667:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658105088 00000100:00100000:1.0:1787784753.035752:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 997 00000100:00100000:1.0:1787784753.035764:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27158:x1874627658105088:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784753.035827:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784753.035859:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d5a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ec4 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.035971:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.035998:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036012:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036029:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036037:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036044:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036128:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036134:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036141:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036430:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784753.036442:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036452:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036466:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784753.036469:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.036474:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036482:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5d61 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036496:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036502:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036508:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036513:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784753.036516:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036522:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036529:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff98951c407900) 00010000:00010000:1.0:1787784753.036531:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.036538:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036544:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5d68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036585:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d5a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ec4 expref: 42 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036594:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784753.036597:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.036600:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5d5a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ec4 expref: 42 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.036609:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784753.036852:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27158:x1874627658105088:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1088us (1368us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784753.036862:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 997 00000100:00100000:3.0:1787784753.055923:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784753.056193:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658105344 00000100:00100000:2.0:1787784753.056207:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658105344 00000100:00100000:2.0:1787784753.056283:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 998 00000100:00100000:2.0:1787784753.056295:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27158:x1874627658105344:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784753.056322:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784753.056349:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5d6f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ecb expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.056663:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.056683:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.056697:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.056713:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.056720:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.056728:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784753.062275:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x16:0x0], index 1: 0 00010000:00010000:3.0:1787784753.078735:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.078758:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.078786:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.078865:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.078872:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.078879:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.078884:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784753.078888:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.078897:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.079017:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:3.0:1787784753.079021:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784753.079027:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.079034:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5d76 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784753.079052:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009dc3254e reply state 0000000023688fa6 transno 4294967939 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784753.079061:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787784753.079113:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc5d6f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.079128:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 43 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784753.079136:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:3.0:1787784753.079139:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784753.079142:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00010000:00010000:0.0:1787784753.079567:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784753.080729:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27158:x1874627658105344:12345-192.168.203.43@tcp:101:cp.0 Request processed in 24425us (24811us total) trans 4294967939 rc 0/0 00000100:00100000:3.0:1787784753.080743:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 998 00010000:00010000:1.0:1787784753.095128:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.095144:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.095190:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.095196:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784753.095200:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.095237:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.095252:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff989530cb1c00) 00010000:00010000:1.0:1787784753.095255:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784753.095260:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.095267:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc5d7d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784753.101661:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784753.101938:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658105472 00000100:00100000:2.0:1787784753.101968:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658105472 00000100:00100000:2.0:1787784753.102044:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 260 00000100:00100000:2.0:1787784753.102055:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658105472:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784753.102072:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784753.102092:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ed2 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784753.102116:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x16:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784753.102135:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=544s v=(0 0) 00010000:00010000:2.0:1787784753.102139:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784753.102184:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.102201:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 11 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.102215:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc000, rc 0) 00000100:00100000:2.0:1787784753.102266:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658105472:12345-192.168.203.43@tcp:101:cp.0 Request processed in 214us (615us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784753.102274:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 260 00000100:00080000:1.0:1787784753.102339:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784753.106442:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784753.106538:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658105600 00000100:00100000:1.0:1787784753.106548:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658105600 00000100:00100000:1.0:1787784753.106608:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 261 00000100:00100000:1.0:1787784753.106617:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:27158:x1874627658105600:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784753.106642:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784753.107411:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ed9 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.107450:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.107460:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784753.107464:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784753.107470:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 12 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.107524:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 12 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.107540:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 12 pid: 6671 timeout: 644 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784753.107555:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668391936:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784753.107734:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953056bcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784753.110674:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784753.112257:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668391936:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784753.112296:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 12 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.112317:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784753.112381:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:27158:x1874627658105600:12345-192.168.203.43@tcp:101:cp.0 Request processed in 5764us (5948us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784753.112391:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 261 00000100:00100000:2.0:1787784753.112995:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658105728 00000100:00100000:2.0:1787784753.113002:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658105728 00010000:00010000:2.0:1787784753.113035:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 12 pid: 6671 timeout: 644 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784753.113078:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 301 00000100:00100000:2.0:1787784753.113088:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658105728:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784753.113119:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784753.113142:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 12 pid: 6671 timeout: 644 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.113195:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 12 pid: 6671 timeout: 644 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.113220:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5d92 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed2 expref: 12 pid: 6671 timeout: 644 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.113236:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784753.113241:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784753.113249:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.113259:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.113296:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784753.113310:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668392064:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784753.113369:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784753.117533:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668392064:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784753.117559:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784753.117627:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658105728:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 4539us (6961us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784753.117637:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 301 00000100:00100000:3.0:1787784753.122536:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784753.122627:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658105984 00000100:00100000:1.0:1787784753.122637:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658105984 00000100:00100000:1.0:1787784753.122704:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 198 00000100:00100000:1.0:1787784753.122716:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:27158:x1874627658105984:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784753.123087:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27158:x1874627658105984:12345-192.168.203.43@tcp:35:cp.0 Request processed in 372us (560us total) trans 4294967940 rc 0/0 00000100:00100000:1.0:1787784753.123096:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 198 00000100:00100000:3.0:1787784753.168084:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784753.168238:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658106112 00000100:00100000:2.0:1787784753.168248:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658106112 00000100:00100000:2.0:1787784753.168320:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 999 00000100:00100000:2.0:1787784753.168330:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:27161:x1874627658106112:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784753.168349:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784753.168368:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5da0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ee0 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168438:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168452:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168463:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168477:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168484:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168490:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168576:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168602:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168609:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784753.168638:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00000004:00200000:2.0:1787784753.168642:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784753.168657:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168668:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168693:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168700:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168706:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168712:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784753.168716:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168723:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168733:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784753.168741:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784753.168746:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168752:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5da7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168767:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168773:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168780:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168785:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784753.168788:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168793:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168800:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff989530cb1c00) 00010000:00010000:2.0:1787784753.168802:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784753.168805:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168824:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5dae lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168852:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5da0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 91 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168872:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.168878:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784753.168881:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784753.168883:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784753.168948:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:27161:x1874627658106112:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 620us (874us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784753.168956:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 999 00000100:00080000:1.0:1787784753.169029:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784753.173633:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784753.173827:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658106240 00000100:00100000:2.0:1787784753.173837:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658106240 00000100:00100000:2.0:1787784753.173914:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 262 00000100:00100000:2.0:1787784753.173924:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658106240:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784753.173942:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784753.173960:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5dbc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ee7 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784753.174016:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668392192:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784753.174099:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d3c0 going to sleep for 11 seconds 00000100:00080000:1.0:1787784753.174355:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1787784753.177577:0:10688:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784753.177585:0:10688:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb mtime: 0 -> 1787784752 00002000:00010000:2.0:1787784753.177588:0:10688:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb atime: 0 -> 1787784752 00002000:00010000:2.0:1787784753.177590:0:10688:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb ctime: 0 -> 1787784752 00000100:00100000:2.0:1787784753.177610:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668392192:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784753.177644:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5dbc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee7 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.177680:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5dbc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee7 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784753.177693:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc000, rc 0) 00000100:00100000:2.0:1787784753.177746:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658106240:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3824us (4122us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784753.177755:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 262 00000100:00100000:0.0:1787784753.181574:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784753.181682:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658106368 00000100:00100000:1.0:1787784753.181692:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658106368 00000100:00100000:1.0:1787784753.181752:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1000 00000100:00100000:1.0:1787784753.181762:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:27161:x1874627658106368:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784753.181783:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784753.181805:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5dc3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0eee expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.181890:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.181903:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.181913:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784753.181967:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 512 00000004:00200000:1.0:1787784753.181971:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784753.181991:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.182018:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5dc3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 92 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.182035:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 91 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.182044:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784753.182061:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784753.182065:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784753.182129:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:27161:x1874627658106368:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 367us (563us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784753.182136:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1000 00000100:00100000:0.0:1787784753.186555:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784753.186639:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658106496 00000100:00100000:2.0:1787784753.186648:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658106496 00000100:00100000:2.0:1787784753.186709:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:2.0:1787784753.186718:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658106496:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784753.186735:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784753.186752:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5dd1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ef5 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784753.186806:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668392320:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784753.186901:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784753.189972:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668392320:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784753.190013:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5dd1 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ef5 expref: 13 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.190036:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4ff5cf7/0xb0ce63137acc5dd1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ef5 expref: 13 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784753.190051:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc000, rc 0) 00000100:00100000:1.0:1787784753.190214:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658106496:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3496us (3667us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784753.190224:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:0.0:1787784753.195219:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784753.195343:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658106752 00000100:00100000:2.0:1787784753.195353:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658106752 00000100:00100000:2.0:1787784753.195414:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1001 00000100:00100000:2.0:1787784753.195425:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:27161:x1874627658106752:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:2.0:1787784753.195711:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].26, 136 off11728 00000100:00100000:2.0:1787784753.195804:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:27161:x1874627658106752:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 380us (592us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784753.196362:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1001 00000100:00100000:1.0:1787784753.649983:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784753.650005:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668392448:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784753.650072:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784753.650130:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668392448 00000100:00100000:1.0:1787784753.650137:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668392448 00000100:00100000:2.0:1787784753.650236:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668392704:0@lo:400:kworker.0 00000100:00100000:1.0:1787784753.650251:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:1.0:1787784753.650259:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668392448:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784753.650267:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784753.650289:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00002000:00000024:1.0:1787784753.650293:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 76480512 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784753.650294:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668392832:0@lo:400:kworker.0 00000100:00100000:2.0:1787784753.650315:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784753.650330:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668392448:12345-0@lo:13:osp-pre-1-0.0 Request processed in 72us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784753.650348:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1787784753.650360:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668392704 00000100:00100000:2.0:1787784753.650364:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668392704 00000100:00080000:1.0:1787784753.650367:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784753.650404:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668392832 00000100:00100000:2.0:1787784753.650407:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668392832 00000100:00100000:3.0:1787784753.650420:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668392960:0@lo:400:kworker.0 00000100:00100000:2.0:1787784753.650434:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1787784753.650439:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668392704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784753.650451:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784753.650471:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784753.650471:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668392704:12345-0@lo:400:kworker.0 Request processed in 32us (202us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784753.650476:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668392576:0@lo:400:kworker.0 00000100:00100000:2.0:1787784753.650478:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1787784753.650490:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:2.0:1787784753.650495:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668392832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784753.650502:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784753.650520:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668392832:12345-0@lo:400:kworker.0 Request processed in 25us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784753.650521:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668393088:0@lo:400:kworker.0 00000100:00100000:2.0:1787784753.650526:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:3.0:1787784753.650541:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1787784753.650544:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784753.650563:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668392576 00000100:00100000:3.0:1787784753.650566:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668392576 00000100:00100000:2.0:1787784753.650568:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668392832:0@lo:400:kworker.0 00000100:00100000:2.0:1787784753.650578:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668392704:0@lo:400:kworker.0 00000100:00100000:3.0:1787784753.650588:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 548 00000100:00100000:3.0:1787784753.650594:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668392576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784753.650605:0:3277:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967353 gen 2 00000004:00080000:2.0:1787784753.650610:0:3277:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967352 committed transno 4294967353 00000100:00100000:3.0:1787784753.650625:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668392576:12345-0@lo:400:kworker.0 Request processed in 32us (122us total) trans 0 rc 0/0 00000004:00080000:2.0:1787784753.650633:0:3277:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401fa4000, transno 4294967353 00000100:00100000:3.0:1787784753.650635:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 548 00000100:00100000:2.0:1787784753.650648:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668392448:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784753.650666:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668392576:0@lo:400:kworker.0 00000100:00100000:1.0:1787784753.650719:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668393088 00000100:00100000:1.0:1787784753.650723:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668393088 00000100:00100000:1.0:1787784753.650759:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:1.0:1787784753.650766:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668393088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784753.650801:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668393088:12345-0@lo:400:kworker.0 Request processed in 36us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784753.650819:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:1.0:1787784753.650837:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668392960 00000100:00100000:1.0:1787784753.650840:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668392960 00000100:00100000:1.0:1787784753.650863:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 311 00000100:00100000:1.0:1787784753.650868:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668392960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784753.650898:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668392960:12345-0@lo:400:kworker.0 Request processed in 29us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784753.650903:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 311 00000100:00100000:1.0:1787784753.650937:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668393088:0@lo:400:kworker.0 00000100:00100000:1.0:1787784753.650946:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668392960:0@lo:400:kworker.0 00000100:00080000:3.0:1787784753.651172:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784753.651176:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:2.0:1787784753.972862:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784753.972870:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784753.972912:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 27 count 4 00000040:00080000:2.0:1787784753.972919:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784753.973215:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa4000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668393216:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784753.973303:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72840 going to sleep for 11 seconds 00000100:00100000:2.0:1787784753.986693:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa4000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668393216:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784753.990779:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784753.990868:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658106880 00000100:00100000:0.0:1787784753.990878:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658106880 00000100:00100000:0.0:1787784753.990935:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1002 00000100:00100000:0.0:1787784753.990944:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27172:x1874627658106880:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784753.990979:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784753.991095:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27172:x1874627658106880:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 150us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784753.991103:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1002 00000040:00080000:0.0:1787784754.003649:0:19633:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784754.003659:0:19633:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784754.003721:0:19633:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 27 count 4 00000040:00080000:0.0:1787784754.003726:0:19633:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784754.005624:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784754.005663:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658107136 00000100:00100000:3.0:1787784754.005672:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658107136 00000100:00100000:3.0:1787784754.005744:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1003 00000100:00100000:3.0:1787784754.005755:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27172:x1874627658107136:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784754.005774:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784754.005912:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5dd8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0efc expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.006008:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5ddf lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.006023:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5ddf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.006036:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5ddf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784754.006047:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:3.0:1787784754.006314:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5dd8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0efc expref: 47 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.006332:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5ddf lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0efc expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.006354:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:3.0:1787784754.006358:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784754.006362:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784754.006416:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27172:x1874627658107136:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 662us (799us total) trans 4294967941 rc 0/0 00000100:00100000:3.0:1787784754.006424:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1003 00000100:00080000:3.0:1787784754.006447:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784754.012508:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784754.012619:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658107264 00000100:00100000:0.0:1787784754.012630:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658107264 00000100:00100000:0.0:1787784754.012688:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:0.0:1787784754.012698:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658107264:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784754.012750:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/1,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.012771:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.012780:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x16:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784754.012784:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784754.012790:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 4/1,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.012835:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 11 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.012850:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 11 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784754.012863:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895e1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668393344:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784754.012944:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f300 going to sleep for 11 seconds 00000100:00100000:0.0:1787784754.016780:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895e1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668393344:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784754.016820:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/1,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784754.023018:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784754.023242:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658107392 00000100:00100000:1.0:1787784754.023252:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658107392 00002000:00100000:1.0:1787784754.023300:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x16:0x0] (0->157) req@ffff98950895e1c0 x1874627658107392/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:635/0 lens 648/0 e 0 to 0 dl 1787784765 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784754.023312:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98950895e1c0 with x1874627658107392 ext(0->157) 00010000:00010000:1.0:1787784754.023321:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 12 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.023351:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 12 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784754.023360:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff98950895e1c0 00000100:00100000:1.0:1787784754.023390:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 10 00000100:00100000:1.0:1787784754.023401:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658107392:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784754.024012:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658107392:12345-192.168.203.43@tcp:4:cp.0 Request processed in 610us (1000us total) trans 4294967354 rc 0/0 00000100:00100000:1.0:1787784754.024021:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 10 00002000:00100000:1.0:1787784754.024028:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x16:0x0] (0->157) req@ffff98950895e1c0 x1874627658107392/t4294967354(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:635/0 lens 648/448 e 0 to 0 dl 1787784765 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784754.024273:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98950895e1c0 with x1874627658107392 ext(0->157) 00010000:00010000:1.0:1787784754.024281:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 14 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.024296:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 14 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784754.024304:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff98950895e1c0 00000100:00100000:3.0:1787784754.040752:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784754.040921:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784754.041018:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658107520 00000100:00100000:2.0:1787784754.041028:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658107520 00000100:00100000:2.0:1787784754.041085:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658107648 00000100:00100000:2.0:1787784754.041090:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658107648 00010000:00010000:2.0:1787784754.041107:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 14 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784754.041129:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 43 00000100:00100000:1.0:1787784754.041182:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2356:x1874627658107520:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784754.041202:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784754.041227:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc5ddf lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0efc expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.041254:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784754.041259:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784754.041310:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2356:x1874627658107520:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 171us (563us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784754.041363:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 302 00000100:00100000:2.0:1787784754.041371:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2354:x1874627658107648:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784754.041381:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784754.041402:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 14 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784754.041423:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 14 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784754.041466:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5d99 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ed9 expref: 14 pid: 6670 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784754.041479:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784754.041484:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784754.041490:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784754.041500:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784754.041695:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784754.041725:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2354:x1874627658107648:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 355us (804us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784754.041733:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 302 00010000:00010000:3.0:1787784754.042532:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042546:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042572:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042580:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042589:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042608:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784754.042612:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042622:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042636:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.042644:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc5ded lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784754.042925:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658107264:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 30226us (30423us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784754.042934:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:1.0:1787784754.047198:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 43 00000100:00100000:1.0:1787784754.048928:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784754.049735:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658107904 00000100:00100000:0.0:1787784754.049764:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658107904 00000100:00100000:0.0:1787784754.049958:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1004 00000100:00100000:0.0:1787784754.049970:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:27172:x1874627658107904:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784754.050095:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:27172:x1874627658107904:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 127us (1173us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784754.050104:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1004 00000100:00100000:3.0:1787784754.063600:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784754.063683:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658108032 00000100:00100000:1.0:1787784754.063693:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658108032 00000100:00100000:1.0:1787784754.063750:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 265 00000100:00100000:1.0:1787784754.063760:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658108032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784754.063777:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784754.063798:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f03 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784754.063836:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x16:0x0] initial LVB size: 158, mtime: 0x6a8f6e30, atime: 0x0, ctime: 0x6a8f6e30, blocks: 0x1 00010000:00010000:1.0:1787784754.063856:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=545s v=(0 0) 00010000:00010000:1.0:1787784754.063860:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784754.063870:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 13 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.063885:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 13 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.063898:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784754.063961:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658108032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 202us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784754.063968:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 265 00000100:00100000:2.0:1787784754.072552:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784754.072644:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658108288 00000100:00100000:0.0:1787784754.072653:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658108288 00000100:00100000:0.0:1787784754.072729:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1005 00000100:00100000:0.0:1787784754.072738:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:27172:x1874627658108288:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784754.072860:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:27172:x1874627658108288:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 122us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784754.072868:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1005 00000100:00100000:0.0:1787784754.077323:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784754.077403:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658108416 00000100:00100000:2.0:1787784754.077413:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658108416 00002000:00100000:2.0:1787784754.077459:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x16:0x0] (0->4095) req@ffff98950895f480 x1874627658108416/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:635/0 lens 488/0 e 0 to 0 dl 1787784765 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784754.077469:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98950895f480 with x1874627658108416 ext(0->4095) 00002000:00010000:2.0:1787784754.077475:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98950895f480 00000100:00100000:2.0:1787784754.077499:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 93 00000100:00100000:2.0:1787784754.077508:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658108416:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784754.077705:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658108416:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 196us (391us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784754.077712:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 93 00002000:00100000:2.0:1787784754.077719:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x16:0x0] (0->4095) req@ffff98950895f480 x1874627658108416/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:635/0 lens 488/4536 e 0 to 0 dl 1787784765 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784754.077728:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98950895f480 with x1874627658108416 ext(0->4095) 00002000:00010000:2.0:1787784754.077731:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98950895f480 00000100:00100000:2.0:1787784754.082475:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784754.082546:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658108544 00000100:00100000:1.0:1787784754.082556:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658108544 00000100:00100000:1.0:1787784754.082616:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1006 00000100:00100000:1.0:1787784754.082627:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:27172:x1874627658108544:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784754.082646:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784754.082666:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5dfb lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f0a expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.082711:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.082725:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.082736:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.082851:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5dfb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.082869:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.082878:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784754.082881:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784754.082884:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784754.082958:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27172:x1874627658108544:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 333us (492us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784754.082965:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1006 00000100:00100000:0.0:1787784754.090977:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784754.091058:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658108672 00000100:00100000:2.0:1787784754.091068:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658108672 00000100:00100000:2.0:1787784754.091126:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1007 00000100:00100000:2.0:1787784754.091137:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27172:x1874627658108672:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784754.091189:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784754.091286:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27172:x1874627658108672:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 151us (318us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784754.091294:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1007 00000100:00100000:3.0:1787784754.096910:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784754.096986:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658108800 00000100:00100000:0.0:1787784754.096996:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658108800 00000100:00100000:0.0:1787784754.097058:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 266 00000100:00100000:0.0:1787784754.097069:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658108800:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784754.097113:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097130:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784754.097135:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784754.097142:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097196:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097226:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097234:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097242:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097249:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784754.097253:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097261:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097275:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784754.097283:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a34f730/0xb0ce63137acc5e09 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784754.097362:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658108800:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 293us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784754.097371:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 266 00010000:00010000:0.0:1787784754.103274:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784754.107602:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784754.107753:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658109056 00000100:00100000:1.0:1787784754.107762:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658109056 00000100:00100000:1.0:1787784754.107826:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1008 00000100:00100000:1.0:1787784754.107837:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27172:x1874627658109056:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784754.107915:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.107931:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.107940:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.107973:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.107987:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 46 pid: 5812 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784754.108002:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f083c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668393472:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784754.108076:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953056bcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784754.110916:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784754.111002:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658109184 00000100:00100000:3.0:1787784754.111011:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658109184 00010000:00010000:3.0:1787784754.111048:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 47 pid: 5812 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784754.111090:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 303 00000100:00100000:3.0:1787784754.111111:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658109184:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784754.111126:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784754.111133:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 47 pid: 5812 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.111178:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 47 pid: 5812 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.111202:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:3.0:1787784754.111207:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.111216:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.111224:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784754.111243:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784754.111289:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658109184:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 180us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784754.111297:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 303 00010000:00010000:1.0:1787784754.111566:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e02 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f0a expref: 46 pid: 5812 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784754.111581:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f083c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668393472:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784754.111607:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111615:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111620:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111803:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111822:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111829:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111835:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784754.111839:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111847:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111856:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784754.111860:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784754.111864:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784754.111870:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5e10 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784754.112053:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27172:x1874627658109056:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 4216us (4458us total) trans 4294967942 rc 0/0 00000100:00100000:1.0:1787784754.112062:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1008 00000040:00080000:3.0:1787784754.112713:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784754.112718:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784754.113254:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1787784754.113263:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784754.123968:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784754.126124:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658109440 00000100:00100000:0.0:1787784754.126137:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658109440 00000100:00100000:0.0:1787784754.126227:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 199 00000100:00100000:0.0:1787784754.126237:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27172:x1874627658109440:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784754.127142:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27172:x1874627658109440:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 903us (3178us total) trans 4294967943 rc 0/0 00000100:00100000:0.0:1787784754.127602:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 199 00010000:00010000:1.0:1787784755.121467:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784755.234364:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967354 is committed 00000040:00080000:3.0:1787784755.867647:0:19685:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784755.867656:0:19685:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784755.867705:0:19685:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 27 count 4 00000040:00080000:3.0:1787784755.867717:0:19685:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784755.952373:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784755.952441:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658109568 00000100:00100000:3.0:1787784755.952450:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658109568 00000100:00100000:3.0:1787784755.952516:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 267 00000100:00100000:3.0:1787784755.952528:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658109568:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784755.952546:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784755.952567:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f11 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784755.952593:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1787784755.952599:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784755.952608:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784755.952627:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784755.952642:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7400, rc 0) 00000100:00100000:3.0:1787784755.952702:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2354:x1874627658109568:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 177us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784755.952709:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 267 00000100:00080000:0.0:1787784755.952852:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784755.972285:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784755.972334:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658109824 00000100:00100000:2.0:1787784755.972344:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658109824 00000100:00100000:2.0:1787784755.972409:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1009 00000100:00100000:2.0:1787784755.972419:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:27181:x1874627658109824:12345-192.168.203.43@tcp:101:lfs.0 00000100:00080000:3.0:1787784755.972437:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784755.972480:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784755.972520:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e1e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f18 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.972720:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5e25 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.972748:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5e25 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.972768:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5e25 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784755.972782:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:2.0:1787784755.972911:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e1e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f18 expref: 94 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.972927:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5e25 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f18 expref: 93 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.972935:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784755.972937:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1787784755.972940:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784755.972990:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:27181:x1874627658109824:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 573us (713us total) trans 4294967944 rc 0/0 00000100:00100000:2.0:1787784755.972997:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1009 00000100:00100000:3.0:1787784755.978258:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784755.978869:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658109952 00000100:00100000:1.0:1787784755.978880:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658109952 00000100:00100000:1.0:1787784755.978942:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 268 00000100:00100000:1.0:1787784755.978953:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658109952:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:1.0:1787784755.978995:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784755.979015:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784755.979025:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784755.979034:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784755.979039:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784755.979044:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784755.979079:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784755.979110:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 13 pid: 10688 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784755.979126:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668393600:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784755.979246:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784755.979257:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 10 pid: 6671 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784755.979267:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668393728:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784755.979291:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f097600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784755.982238:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668393600:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784755.982256:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668393728:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784755.982281:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784755.987407:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784755.987452:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784755.987486:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658110208 00000100:00100000:0.0:1787784755.987495:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658110208 00010000:00010000:0.0:1787784755.987543:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 11 pid: 6671 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784755.987572:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658110080 00000100:00100000:0.0:1787784755.987575:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658110080 00010000:00010000:0.0:1787784755.987588:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 14 pid: 10688 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784755.987634:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:0.0:1787784755.987643:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658110208:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784755.987657:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1787784755.987675:0:5795:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:0.0:1787784755.987681:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 11 pid: 6671 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987694:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 11 pid: 6671 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987716:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000090185ac7/0xb0ce63137acc5df4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f03 expref: 11 pid: 6671 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987732:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784755.987782:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658110208:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 140us (384us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784755.987801:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:0.0:1787784755.987825:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:0.0:1787784755.987832:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2356:x1874627658110080:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784755.987840:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784755.987848:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 14 pid: 10688 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987870:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 14 pid: 10688 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987885:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b91994e3/0xb0ce63137acc5e17 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f11 expref: 14 pid: 10688 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987896:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784755.987901:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784755.987907:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987916:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.987944:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784755.987967:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658110080:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 136us (514us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784755.987975:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 305 00010000:00010000:0.0:1787784755.988029:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988039:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988063:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988078:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988087:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988094:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784755.988099:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988109:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988122:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784755.988130:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc5e33 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784755.988197:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658109952:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 9246us (9947us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784755.988205:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 268 00000100:00080000:0.0:1787784755.988226:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784755.994261:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784755.994356:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658110464 00000100:00100000:2.0:1787784755.994366:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658110464 00000100:00100000:2.0:1787784755.994428:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 200 00000100:00100000:2.0:1787784755.994438:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d4050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:27181:x1874627658110464:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:2.0:1787784755.994498:0:5814:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x1b:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5e25 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f18 expref: 93 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784755.994699:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967945 00000001:00080000:2.0:1787784755.995331:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967946 00010000:00010000:2.0:1787784755.995390:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.995406:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.995414:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 96 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.995422:0:5814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 5/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.995454:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 96 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.995469:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 96 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784755.995483:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa4780 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668393856:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784755.995571:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784755.995593:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 46 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784755.995602:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668393984:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784755.995628:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784756.005199:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.005226:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784756.006660:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa4780 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668393856:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784756.006680:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668393984:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784756.006712:0:5814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.006829:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658110592 00000100:00100000:0.0:1787784756.006847:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658110592 00010000:00010000:0.0:1787784756.006882:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 97 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.006917:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658110720 00000100:00100000:0.0:1787784756.006920:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658110720 00010000:00010000:0.0:1787784756.006951:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 47 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787784756.006966:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784756.006979:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00100000:0.0:1787784756.006990:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:2356:x1874627658110592:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784756.007005:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784756.007011:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 97 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007025:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 97 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007047:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5db5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ee0 expref: 97 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007063:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784756.007068:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007077:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007084:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.007173:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2356:x1874627658110592:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 183us (1980us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.007183:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00100000:0.0:1787784756.007208:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 307 00000100:00100000:0.0:1787784756.007216:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658110720:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784756.007225:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784756.007232:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 47 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007244:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 47 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007258:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5d84 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ecb expref: 47 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007270:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784756.007274:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007282:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007291:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.007329:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.007356:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2356:x1874627658110720:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 140us (2129us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.007363:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 307 00010000:00010000:3.0:1787784756.007693:0:5814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.007713:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784756.007833:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967947 00000001:00080000:3.0:1787784756.008110:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967948 00000001:00080000:3.0:1787784756.008201:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967949 00000100:00100000:3.0:1787784756.008287:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668394112:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1787784756.008370:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784756.009507:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051dcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784756.013229:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668394112 00000100:00100000:1.0:1787784756.013243:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668394112 00000100:00080000:3.0:1787784756.017619:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784756.024334:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:0.0:1787784756.024360:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668394112:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1787784756.024502:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668394112:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 154us (16133us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.024511:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1787784756.025869:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668394112:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1787784756.026464:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967950 00000100:00100000:2.0:1787784756.026554:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668394240:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1787784756.026610:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784756.035543:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72900 going to sleep for 11 seconds 00000100:00100000:1.0:1787784756.039225:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668394240 00000100:00100000:1.0:1787784756.039239:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668394240 00000100:00100000:1.0:1787784756.039297:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:1.0:1787784756.039310:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668394240:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:1.0:1787784756.039654:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5814:x1874627668394240:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 347us (13042us total) trans 4294967355 rc 0/0 00000100:00100000:1.0:1787784756.039662:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000100:00100000:0.0:1787784756.042377:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668394240:0@lo:1000:mdt_rdpg00_000.0 00010000:00010000:0.0:1787784756.042412:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.042424:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.042432:0:5814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.042439:0:5814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784756.042443:0:5814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.042452:0:5814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.042468:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784756.042472:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1787784756.042477:0:5814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.042484:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5e3a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784756.042734:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x3f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1787784756.042876:0:5814:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].13, 64 off9096 00000004:00080000:0.0:1787784756.042888:0:5814:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 547129739851ns, next at 548129737735ns, rc = 0 00000001:00080000:0.0:1787784756.042897:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967951 00010000:00010000:0.0:1787784756.042944:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784756.042947:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784756.042954:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5e25 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f18 expref: 100 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.043069:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d4050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:27181:x1874627658110464:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 48632us (48817us total) trans 4294967952 rc 0/0 00000100:00100000:0.0:1787784756.043079:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 200 00000020:00080000:0.0:1787784756.043085:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:1.0:1787784756.068927:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784756.069804:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658110848 00000100:00100000:2.0:1787784756.069815:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658110848 00000100:00100000:2.0:1787784756.069877:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1010 00000100:00100000:2.0:1787784756.069888:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:27182:x1874627658110848:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:2.0:1787784756.069913:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784756.069957:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5e41 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f1f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.070022:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.070045:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.070056:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.070070:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.070077:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.070083:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070237:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e56 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070250:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e56 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070260:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e56 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784756.070298:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00000004:00200000:1.0:1787784756.070302:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784756.070322:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e56 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070334:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e56 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070342:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070348:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070355:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070361:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784756.070365:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070371:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070396:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:1.0:1787784756.070404:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784756.070409:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070416:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5e48 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070429:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070435:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070440:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070445:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784756.070448:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070454:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070461:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff989530cb1e00) 00010000:00010000:1.0:1787784756.070464:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784756.070466:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070472:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc5e4f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070506:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc5e41 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f1f expref: 101 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070528:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e56 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f1f expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.070536:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98951c407a00) 00010000:00010000:1.0:1787784756.070538:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784756.070541:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d800, rc 0) 00000100:00100000:1.0:1787784756.070611:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:27182:x1874627658110848:12345-192.168.203.43@tcp:101:diff.0 Request processed in 724us (1691us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784756.070619:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1010 00000100:00100000:1.0:1787784756.076717:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.076789:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658111104 00000100:00100000:3.0:1787784756.076807:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658111104 00000100:00100000:3.0:1787784756.076864:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1011 00000100:00100000:3.0:1787784756.076874:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:27182:x1874627658111104:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:3.0:1787784756.076895:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784756.076915:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e5d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f26 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.077118:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5e64 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.077133:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5e64 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.077210:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5e64 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784756.077223:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:3.0:1787784756.077404:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f26 expref: 103 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.077435:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5e64 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f26 expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.077445:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98951c407a00) 00010000:00010000:3.0:1787784756.077448:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784756.077452:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784756.079964:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:27182:x1874627658111104:12345-192.168.203.43@tcp:101:diff.0 Request processed in 3081us (3244us total) trans 4294967953 rc 0/0 00000100:00100000:3.0:1787784756.079978:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1011 00000100:00100000:2.0:1787784756.089932:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.090081:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658111360 00000100:00100000:3.0:1787784756.090091:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658111360 00000100:00100000:0.0:1787784756.090195:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 44 00000100:00100000:0.0:1787784756.090210:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:27182:x1874627658111360:12345-192.168.203.43@tcp:103:diff.0 00010000:00010000:0.0:1787784756.090228:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1787784756.090256:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5e64 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f26 expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.090276:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e56 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f1f expref: 101 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.090286:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff98951c407a00) 00010000:00010000:0.0:1787784756.090297:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.090355:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:27182:x1874627658111360:12345-192.168.203.43@tcp:103:diff.0 Request processed in 149us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.090372:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 44 00000100:00100000:1.0:1787784756.095772:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784756.095910:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658111488 00000100:00100000:2.0:1787784756.095920:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658111488 00000100:00100000:2.0:1787784756.095975:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1012 00000100:00100000:2.0:1787784756.095985:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:27182:x1874627658111488:12345-192.168.203.43@tcp:58:diff.0 00010000:00010000:2.0:1787784756.096089:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.096105:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.096116:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784756.096320:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].27, 136 off11864 00000100:00100000:2.0:1787784756.096423:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:27182:x1874627658111488:12345-192.168.203.43@tcp:58:diff.0 Request processed in 439us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784756.096432:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1012 00000100:00100000:3.0:1787784756.101427:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.101464:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658111616 00000100:00100000:1.0:1787784756.101473:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658111616 00000100:00100000:1.0:1787784756.101547:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1013 00000100:00100000:1.0:1787784756.101558:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:27182:x1874627658111616:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:1.0:1787784756.101579:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784756.101598:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f2d expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.101655:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.101667:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f2d expref: 101 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.101697:0:10850:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.101721:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f2d expref: 101 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.101733:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff989530cb1300) 00010000:00010000:1.0:1787784756.101736:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784756.101738:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f2d expref: 101 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.101744:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d200, rc 0) 00000100:00100000:1.0:1787784756.101787:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:27182:x1874627658111616:12345-192.168.203.43@tcp:101:diff.0 Request processed in 232us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784756.101806:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1013 00010000:00010000:0.0:1787784756.145347:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784756.146122:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784756.146134:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784756.146225:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 28 count 5 00000040:00080000:3.0:1787784756.146231:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784756.146345:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133840 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668394368:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784756.146446:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784756.163247:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.163347:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658111744 00000100:00100000:1.0:1787784756.163356:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658111744 00000100:00100000:1.0:1787784756.163416:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1014 00000100:00100000:1.0:1787784756.163427:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658111744:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784756.163468:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784756.163566:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658111744:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 140us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784756.163574:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1014 00000100:00100000:2.0:1787784756.165741:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133840 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668394368:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784756.167445:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.168501:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658111872 00000100:00100000:3.0:1787784756.168514:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658111872 00000100:00100000:3.0:1787784756.168574:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1015 00000100:00100000:3.0:1787784756.168585:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27183:x1874627658111872:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1787784756.168657:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168710:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168724:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168742:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168751:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168758:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168785:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168810:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168818:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784756.168855:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00000004:00200000:3.0:1787784756.168860:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784756.168872:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168884:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168902:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168909:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168916:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784756.168921:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168929:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168942:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784756.168945:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784756.168950:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168957:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5e80 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168971:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168977:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168984:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.168989:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784756.168998:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169006:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169014:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x2b74b2f9 (ffff989505542f00) 00010000:00010000:3.0:1787784756.169017:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784756.169020:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169027:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5e87 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169042:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169049:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169056:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169061:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784756.169064:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169070:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169079:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff989530cb1300) 00010000:00010000:3.0:1787784756.169081:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784756.169090:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.169097:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5e8e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784756.169208:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27183:x1874627658111872:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 624us (1769us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.169217:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1015 00000100:00100000:0.0:1787784756.173198:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.173275:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658112128 00000100:00100000:1.0:1787784756.173285:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658112128 00000100:00100000:1.0:1787784756.173338:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1016 00000100:00100000:1.0:1787784756.173347:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27183:x1874627658112128:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784756.173387:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784756.173409:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e95 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f34 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173518:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173534:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173544:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784756.173557:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:1.0:1787784756.173649:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173657:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173664:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173685:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784756.173690:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173697:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173708:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff989530cb1300) 00010000:00010000:1.0:1787784756.173717:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784756.173721:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173726:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5e9c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173767:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e95 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f34 expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.173777:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784756.173779:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784756.173783:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5e95 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f34 expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.174064:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d800, rc 0) 00000100:00100000:1.0:1787784756.174130:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658112128:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 784us (942us total) trans 4294967954 rc 301/301 00000100:00100000:1.0:1787784756.174138:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1016 00000100:00100000:2.0:1787784756.178907:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.179258:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658112256 00000100:00100000:3.0:1787784756.179268:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658112256 00000100:00100000:3.0:1787784756.179363:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1017 00000100:00100000:3.0:1787784756.179384:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658112256:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784756.179420:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784756.179523:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658112256:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 150us (624us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.179531:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1017 00000100:00100000:2.0:1787784756.189215:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.189580:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658112512 00000100:00100000:1.0:1787784756.189591:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658112512 00000100:00100000:1.0:1787784756.189652:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1018 00000100:00100000:1.0:1787784756.189663:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658112512:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784756.189683:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784756.189708:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5eaa lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f3b expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.191508:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5eb1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.191527:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5eb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.191539:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5eb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784756.191549:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1c:0x0]: returning size 0 00010000:00010000:0.0:1787784756.191597:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc5eaa lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f3b expref: 50 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.191614:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5eb1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f3b expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.191623:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff98950a162d00) 00010000:00010000:0.0:1787784756.191626:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784756.191629:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030e00, rc 0) 00000100:00100000:0.0:1787784756.191746:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:27183:x1874627658112512:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2085us (4773us total) trans 4294967955 rc 0/0 00000100:00100000:0.0:1787784756.191755:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1018 00000100:00100000:0.0:1787784756.198922:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.199016:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658112768 00000100:00100000:1.0:1787784756.199038:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658112768 00000100:00100000:1.0:1787784756.199252:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1019 00000100:00100000:1.0:1787784756.199264:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:27183:x1874627658112768:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784756.199325:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784756.199353:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5eb1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f3b expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.199384:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff98950a162d00) 00010000:00010000:1.0:1787784756.199396:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784756.199422:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.199438:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 100 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.199449:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 4/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.199484:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 100 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.199498:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 100 pid: 10850 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784756.199513:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668394496:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784756.199602:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f097240 going to sleep for 11 seconds 00000100:00100000:2.0:1787784756.203709:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.203799:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658112896 00000100:00100000:0.0:1787784756.203809:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658112896 00010000:00010000:0.0:1787784756.203861:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 101 pid: 10850 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.203902:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 308 00000100:00100000:0.0:1787784756.203912:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:2357:x1874627658112896:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784756.203926:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784756.203933:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 101 pid: 10850 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.203944:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 101 pid: 10850 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.203964:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:0.0:1787784756.203969:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.203978:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.203985:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.204002:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.204055:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:2357:x1874627658112896:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 144us (354us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.204063:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 308 00010000:00010000:1.0:1787784756.205745:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5dca lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0eee expref: 100 pid: 10850 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784756.205935:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668394496:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784756.205957:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.206056:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.206063:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.208921:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.208947:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.208959:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209041:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209049:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209055:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209060:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784756.209064:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209072:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209084:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:0.0:1787784756.209087:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784756.209103:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209109:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc5ebf lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209140:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209657:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209665:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209670:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784756.209674:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209680:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209690:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x807fa39d (ffff98950a162b00) 00010000:00010000:0.0:1787784756.209692:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784756.209696:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.209702:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5ec6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0x807fa39d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.209899:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:27183:x1874627658112768:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 10633us (10982us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784756.209909:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1019 00000100:00100000:3.0:1787784756.219309:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.219352:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658113024 00000100:00100000:3.0:1787784756.219361:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658113024 00000100:00100000:3.0:1787784756.219413:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1020 00000100:00100000:3.0:1787784756.219425:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:27183:x1874627658113024:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784756.219467:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784756.219899:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5ecd lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f42 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.219941:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.219972:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.219987:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.220116:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc5ecd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.220137:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 49 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.220182:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784756.220186:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784756.220191:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784756.220251:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:27183:x1874627658113024:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 829us (951us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.220260:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1020 00000100:00100000:3.0:1787784756.223711:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.223754:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658113280 00000100:00100000:3.0:1787784756.223763:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658113280 00000100:00100000:3.0:1787784756.223832:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1021 00000100:00100000:3.0:1787784756.223842:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:27183:x1874627658113280:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784756.223922:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.223939:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.223965:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784756.224212:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001d835ec5 reply state 00000000c5dc6e7b transno 4294967956 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784756.224221:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784756.224301:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:27183:x1874627658113280:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 459us (598us total) trans 4294967956 rc 0/0 00000100:00100000:3.0:1787784756.224308:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1021 00010000:00010000:2.0:1787784756.228031:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.228052:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784756.228064:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:2.0:1787784756.228068:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.228076:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784756.228082:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.228092:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.228124:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:2.0:1787784756.228128:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787784756.228133:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.228141:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc5edb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784756.228194:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658113408 00000100:00100000:1.0:1787784756.228205:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658113408 00000100:00100000:1.0:1787784756.228263:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1022 00000100:00100000:1.0:1787784756.228273:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:27183:x1874627658113408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784756.228298:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784756.228330:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f49 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.228405:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.228419:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.228428:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:1.0:1787784756.228431:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784756.228434:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c400, rc 0) 00000100:00100000:1.0:1787784756.228491:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:27183:x1874627658113408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 220us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784756.228498:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1022 00000100:00100000:2.0:1787784756.243127:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.243253:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658113536 00000100:00100000:0.0:1787784756.243267:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658113536 00000100:00100000:0.0:1787784756.243330:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 201 00000100:00100000:0.0:1787784756.243361:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:27183:x1874627658113536:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:0.0:1787784756.243531:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:27183:x1874627658113536:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 173us (417us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.243540:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 201 00000100:00100000:2.0:1787784756.265947:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.266308:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658113792 00000100:00100000:0.0:1787784756.266324:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658113792 00000100:00100000:0.0:1787784756.266389:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 202 00000100:00100000:0.0:1787784756.266402:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:27183:x1874627658113792:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784756.266710:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:27183:x1874627658113792:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 307us (785us total) trans 4294967957 rc 0/0 00000100:00100000:0.0:1787784756.266721:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 202 00000100:00100000:0.0:1787784756.269572:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.269672:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658114048 00000100:00100000:3.0:1787784756.269685:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658114048 00000100:00100000:3.0:1787784756.269760:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1023 00000100:00100000:3.0:1787784756.269773:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+52:27183:x1874627658114048:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784756.269935:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+52:27183:x1874627658114048:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 164us (376us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.269947:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1023 00000100:00100000:3.0:1787784756.279898:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784756.281677:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658114304 00000100:00100000:2.0:1787784756.281697:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658114304 00000100:00100000:2.0:1787784756.281778:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1024 00000100:00100000:2.0:1787784756.281800:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:27183:x1874627658114304:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784756.281828:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784756.281857:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5ee9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f50 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.283520:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.283551:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.283563:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.283610:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.283628:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 53 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.283643:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668394624:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784756.283758:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72600 going to sleep for 11 seconds 00000001:00080000:0.0:1787784756.287627:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967955 is committed 00000001:00080000:0.0:1787784756.287654:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967952 is committed 00000004:00080000:0.0:1787784756.287658:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 14 committed 00000020:00080000:0.0:1787784756.287705:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989538a72780 refcount 2 committed 0 result 0 batchid 17 00000020:00080000:0.0:1787784756.287711:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895053f0e00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989538a72cc0 00000020:00080000:0.0:1787784756.287714:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895053f0100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895393c1000 00000020:00080000:0.0:1787784756.287741:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 16 batchid updated 00000001:00080000:0.0:1787784756.287747:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967953 is committed 00000020:00080000:2.0:1787784756.288788:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 17 committed batchid 16 00000020:00080000:2.0:1787784756.289019:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 16 00000100:00100000:2.0:1787784756.289033:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784756.289058:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668394752:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787784756.289305:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784756.290015:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668394752 00000100:00100000:3.0:1787784756.290029:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668394752 00000100:00100000:3.0:1787784756.290261:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:3.0:1787784756.290275:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668394752:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1787784756.290316:0:6669:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:22 count 1 00002000:00010000:3.0:1787784756.290361:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x16:0x0] initial LVB size: 158, mtime: 0x6a8f6e30, atime: 0x0, ctime: 0x6a8f6e30, blocks: 0xb 00010000:00010000:3.0:1787784756.290371:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.290481:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=547s v=(0 0) 00010000:00010000:3.0:1787784756.290487:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784756.290495:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.290513:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.290524:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.290533:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.290607:0:6669:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784756.291558:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3279:x1874627668394752:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1280us (2247us total) trans 4294967356 rc 0/0 00000100:00100000:3.0:1787784756.291571:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000004:00080000:0.0:1787784756.291769:0:3279:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402ade580/2, rc 0, transno 4294967356 00000100:00100000:0.0:1787784756.291777:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668394752:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787784756.297491:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668394624:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784756.297538:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784756.297586:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:2.0:1787784756.298206:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784756.298220:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.298232:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784756.298238:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.298251:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.298279:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.298289:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc5ef7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.298316:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784756.298319:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784756.298320:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784756.299276:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658114432 00000100:00100000:0.0:1787784756.299293:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658114432 00010000:00010000:0.0:1787784756.299354:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 47 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.299407:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 309 00000100:00100000:0.0:1787784756.299419:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658114432:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784756.299438:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784756.299445:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 47 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.299459:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 47 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.299485:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5ee2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f49 expref: 47 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.299503:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:0.0:1787784756.299510:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.299520:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.299529:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.299562:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.299664:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2356:x1874627658114432:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 247us (2078us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.299674:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 309 00010000:00010000:2.0:1787784756.300130:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300172:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784756.300196:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1c:0x0]: returning size 0 00000004:00080000:2.0:1787784756.300570:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x14:0x0], index 0: 0 00010000:00010000:2.0:1787784756.300740:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300750:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300757:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300763:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784756.300768:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300776:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300788:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:2.0:1787784756.300801:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784756.300805:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300811:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5ef0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300872:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5ee9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f50 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300894:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff989530cb1400) 00010000:00010000:2.0:1787784756.300897:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784756.300902:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5ee9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f50 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.300911:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784756.300967:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:27183:x1874627658114304:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 19179us (21079us total) trans 4294967958 rc 301/301 00000100:00100000:2.0:1787784756.300977:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1024 00000100:00100000:1.0:1787784756.308915:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.309510:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658114688 00000100:00100000:3.0:1787784756.309520:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658114688 00000100:00100000:3.0:1787784756.309619:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 203 00000100:00100000:3.0:1787784756.309630:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:27183:x1874627658114688:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784756.309991:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27183:x1874627658114688:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 361us (1084us total) trans 4294967959 rc 0/0 00000100:00100000:3.0:1787784756.309999:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 203 00000100:00100000:0.0:1787784756.315524:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.315633:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658114816 00000100:00100000:3.0:1787784756.315643:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658114816 00000100:00100000:3.0:1787784756.315727:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1025 00000100:00100000:3.0:1787784756.315737:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:27183:x1874627658114816:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784756.315758:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784756.315778:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f05 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f57 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.315874:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5f0c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.315887:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5f0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.315899:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5f0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784756.315933:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1c:0x0]: returning size 0 00000004:00200000:3.0:1787784756.315937:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784756.315955:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5f0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.315981:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f05 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f57 expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.315999:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5f0c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f57 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.316007:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784756.316010:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1787784756.316027:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784756.316088:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658114816:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 352us (572us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.316678:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1025 00000100:00100000:1.0:1787784756.322056:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.322437:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658114944 00000100:00100000:3.0:1787784756.322446:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658114944 00000100:00100000:2.0:1787784756.322657:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 270 00000100:00100000:2.0:1787784756.322668:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658114944:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784756.322686:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784756.322707:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f5e expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784756.322733:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784756.322753:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=547s v=(0 0) 00010000:00010000:2.0:1787784756.322757:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784756.322767:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.322781:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.322840:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784756.322904:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658114944:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 237us (855us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784756.322912:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 270 00000100:00080000:3.0:1787784756.326311:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784756.326551:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.326891:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658115200 00000100:00100000:3.0:1787784756.326900:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658115200 00000100:00100000:3.0:1787784756.326996:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1026 00000100:00100000:3.0:1787784756.327006:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658115200:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784756.327184:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658115200:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 180us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.327192:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1026 00000100:00100000:1.0:1787784756.336480:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.337299:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658115328 00000100:00100000:0.0:1787784756.337310:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658115328 00000100:00100000:0.0:1787784756.337373:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:0.0:1787784756.337384:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:27183:x1874627658115328:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784756.337402:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784756.337423:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f65 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.337455:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.337466:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784756.337471:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784756.337476:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.337514:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.337527:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 10 pid: 6670 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.337541:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668394880:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784756.337756:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a32840 going to sleep for 11 seconds 00000100:00100000:1.0:1787784756.345998:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.346108:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668394880:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784756.346142:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.346192:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030c00, rc 0) 00000100:00100000:0.0:1787784756.346239:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:27183:x1874627658115328:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 8857us (9769us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.346249:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:2.0:1787784756.346761:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658115456 00000100:00100000:2.0:1787784756.346769:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658115456 00010000:00010000:2.0:1787784756.346866:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 10 pid: 6670 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.346902:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00100000:2.0:1787784756.346912:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658115456:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784756.346924:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784756.346943:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 10 pid: 6670 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.346957:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 10 pid: 6670 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.346979:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078cc824e/0xb0ce63137acc5f13 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f5e expref: 10 pid: 6670 timeout: 647 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.346993:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784756.346997:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784756.347005:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.347030:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.347077:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.347264:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adf480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668395008:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784756.347335:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784756.352522:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668395008:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784756.352552:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.352607:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658115456:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5696us (6619us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.352619:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00100000:1.0:1787784756.352696:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.353033:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658115584 00000100:00100000:0.0:1787784756.353040:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658115584 00002000:00100000:0.0:1787784756.353336:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x14:0x0] (0->157) req@ffff989530504000 x1874627658115584/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:637/0 lens 648/0 e 0 to 0 dl 1787784767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784756.353348:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530504000 with x1874627658115584 ext(0->157) 00002000:00010000:0.0:1787784756.353354:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989530504000 00000100:00100000:0.0:1787784756.353384:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 94 00000100:00100000:0.0:1787784756.353394:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658115584:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1787784756.356417:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658115584:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 3016us (3712us total) trans 4294967348 rc 0/0 00000100:00100000:0.0:1787784756.356446:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 94 00002000:00100000:0.0:1787784756.356458:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x14:0x0] (0->157) req@ffff989530504000 x1874627658115584/t4294967348(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:637/0 lens 648/448 e 0 to 0 dl 1787784767 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784756.356468:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530504000 with x1874627658115584 ext(0->157) 00002000:00010000:0.0:1787784756.356474:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989530504000 00000100:00100000:1.0:1787784756.360742:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.360842:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658115712 00000100:00100000:0.0:1787784756.360852:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658115712 00000100:00100000:0.0:1787784756.360911:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1027 00000100:00100000:0.0:1787784756.360921:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:27183:x1874627658115712:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1787784756.404952:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967957 is committed 00000001:00080000:2.0:1787784756.404983:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967958 is committed 00000020:00080000:2.0:1787784756.404991:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 17 batchid updated 00000020:00080000:2.0:1787784756.405005:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 17 00000020:00080000:2.0:1787784756.405019:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 17 00000001:00080000:2.0:1787784756.405034:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967959 is committed 00000100:00100000:0.0:1787784756.405290:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:27183:x1874627658115712:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 44366us (44553us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.405301:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1027 00000100:00100000:1.0:1787784756.408272:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.408356:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658115840 00000100:00100000:3.0:1787784756.408366:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658115840 00000100:00100000:3.0:1787784756.408425:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 272 00000100:00100000:3.0:1787784756.408436:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658115840:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1787784756.452707:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658115840:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 44262us (44433us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784756.452721:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 272 00000001:00080000:1.0:1787784756.453108:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967348 is committed 00000100:00100000:2.0:1787784756.460650:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.461039:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658116096 00000100:00100000:0.0:1787784756.461050:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658116096 00000100:00100000:0.0:1787784756.461120:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1028 00000100:00100000:0.0:1787784756.461131:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:27183:x1874627658116096:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784756.461221:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784756.461248:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc5f0c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f57 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.461267:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:0.0:1787784756.461273:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784756.461302:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.461333:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.461343:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784756.461545:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a83ed5be reply state 000000002254abaa transno 4294967960 ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.461556:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784756.461648:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27183:x1874627658116096:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 518us (1007us total) trans 4294967960 rc 0/0 00000100:00100000:0.0:1787784756.461656:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1028 00010000:00010000:1.0:1787784756.464286:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.464304:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.464311:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.464318:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784756.464329:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.464337:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.464354:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:1.0:1787784756.464357:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784756.464362:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.464368:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f21 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.475219:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.475275:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658116224 00000100:00100000:3.0:1787784756.475284:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658116224 00000100:00100000:0.0:1787784756.476463:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1029 00000100:00100000:0.0:1787784756.476479:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:27183:x1874627658116224:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784756.476505:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784756.476528:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f6c expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.476590:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.476603:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.476620:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:0.0:1787784756.476622:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784756.476639:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:0.0:1787784756.476705:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27183:x1874627658116224:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 232us (1497us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.476713:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1029 00000100:00100000:3.0:1787784756.482220:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.482431:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658116352 00000100:00100000:0.0:1787784756.482441:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658116352 00000100:00100000:0.0:1787784756.482500:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 273 00000100:00100000:0.0:1787784756.482510:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658116352:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1787784756.482761:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:0.0:1787784756.482765:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 0 -> 1787784752 00002000:00010000:0.0:1787784756.482767:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb atime from disk: 0 -> 1787784752 00002000:00010000:0.0:1787784756.482769:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 0 -> 1787784752 00002000:00010000:0.0:1787784756.482771:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1787784756.482943:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658116352:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 433us (732us total) trans 4294967349 rc 0/0 00000100:00100000:0.0:1787784756.482951:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 273 00000100:00100000:1.0:1787784756.508277:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.508466:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658116608 00000100:00100000:0.0:1787784756.508476:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658116608 00000100:00100000:0.0:1787784756.508541:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1030 00000100:00100000:0.0:1787784756.508552:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:27183:x1874627658116608:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1787784756.508619:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.508637:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.508647:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.508705:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.508720:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 44 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.508733:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668395136:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784756.508883:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953079a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784756.517023:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668395136:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784756.517056:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784756.520533:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.520915:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658116736 00000100:00100000:0.0:1787784756.520925:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658116736 00010000:00010000:0.0:1787784756.520971:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 45 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.521052:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 311 00000100:00100000:0.0:1787784756.521131:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2357:x1874627658116736:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784756.521200:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784756.521208:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 45 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.521222:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 45 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.521264:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc5ed4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f42 expref: 45 pid: 5812 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.521280:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:0.0:1787784756.521286:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.521296:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.521303:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.521339:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.521391:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658116736:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 330us (865us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.521399:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 311 00010000:00010000:2.0:1787784756.521867:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521876:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521903:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521917:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521924:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521930:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784756.521933:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521940:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521950:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:2.0:1787784756.521953:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787784756.521957:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.521963:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f2f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.522001:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:27183:x1874627658116608:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 13451us (13733us total) trans 0 rc -22/-22 00000100:00100000:2.0:1787784756.522008:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1030 00000100:00100000:2.0:1787784756.537267:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.537332:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658116864 00000100:00100000:0.0:1787784756.537344:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658116864 00000100:00100000:0.0:1787784756.537404:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 274 00000100:00100000:0.0:1787784756.537415:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658116864:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784756.537463:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.537487:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.537515:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784756.537520:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784756.537525:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.537574:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.537588:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 10 pid: 6671 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.537604:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668395264:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784756.537686:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953079a480 going to sleep for 11 seconds 00000100:00100000:0.0:1787784756.540039:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668395264:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784756.540080:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.541624:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.541727:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658116992 00000100:00100000:0.0:1787784756.541736:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658116992 00010000:00010000:0.0:1787784756.541775:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 11 pid: 6671 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.541834:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 312 00000100:00100000:0.0:1787784756.541845:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658116992:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784756.541877:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784756.541907:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 11 pid: 6671 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.541921:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 11 pid: 6671 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.541943:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a34f730/0xb0ce63137acc5f1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f65 expref: 11 pid: 6671 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.541962:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784756.541967:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784756.541973:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.541986:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.542024:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784756.542096:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658116992:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 253us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.542104:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 312 00010000:00010000:1.0:1787784756.542227:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542237:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542264:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542273:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542282:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542289:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784756.542294:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542303:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542316:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.542333:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034a887f2/0xb0ce63137acc5f36 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784756.542397:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658116864:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 4984us (5140us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784756.542406:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 274 00000100:00100000:2.0:1787784756.546952:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.547073:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658117248 00000100:00100000:3.0:1787784756.547084:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658117248 00000100:00100000:3.0:1787784756.547860:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1031 00000100:00100000:3.0:1787784756.547872:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:27183:x1874627658117248:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784756.547951:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.548171:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.548185:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.548389:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.548401:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.548408:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.548439:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.548452:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 44 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784756.548467:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668395392:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784756.548549:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953056b0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784756.550839:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668395392:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784756.550864:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.554285:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784756.554599:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658117376 00000100:00100000:3.0:1787784756.554609:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658117376 00010000:00010000:3.0:1787784756.554650:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 45 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784756.554694:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 313 00000100:00100000:3.0:1787784756.554717:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2357:x1874627658117376:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784756.554734:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784756.554740:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 45 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.554754:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 45 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.554777:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f28 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f6c expref: 45 pid: 5813 timeout: 647 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.554808:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:3.0:1787784756.554813:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.554821:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.554829:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784756.554860:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784756.554906:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658117376:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 191us (632us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.554914:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 313 00010000:00010000:2.0:1787784756.555338:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.555357:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784756.555551:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967961 00000001:00080000:2.0:1787784756.555696:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967962 00000001:00080000:2.0:1787784756.555742:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967963 00000100:00100000:2.0:1787784756.555837:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adfc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668395520:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787784756.555888:0:5813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784756.555917:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f780 going to sleep for 11 seconds 00000100:00100000:3.0:1787784756.556205:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668395520 00000100:00100000:3.0:1787784756.556211:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668395520 00000100:00100000:3.0:1787784756.556268:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:3.0:1787784756.556276:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668395520:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787784756.556356:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668395520:12345-0@lo:1000:mdt00_002.0 Request processed in 82us (468us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.556364:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000100:00080000:2.0:1787784756.557181:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784756.557222:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adfc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668395520:0@lo:1000:mdt00_002.0 00000001:00080000:2.0:1787784756.557572:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967964 00000100:00100000:2.0:1787784756.557629:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adfc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668395648:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787784756.557667:0:5813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784756.557694:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f540 going to sleep for 11 seconds 00000100:00100000:3.0:1787784756.558046:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668395648 00000100:00100000:3.0:1787784756.558053:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668395648 00000100:00100000:3.0:1787784756.558143:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:3.0:1787784756.558177:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668395648:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787784756.558355:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5813:x1874627668395648:12345-0@lo:1000:mdt00_002.0 Request processed in 180us (687us total) trans 4294967350 rc 0/0 00000100:00100000:3.0:1787784756.558362:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00000100:00100000:2.0:1787784756.558440:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adfc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668395648:0@lo:1000:mdt00_002.0 00000001:00080000:2.0:1787784756.558514:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967965 00010000:00010000:2.0:1787784756.558562:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558572:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558580:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558586:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784756.558590:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558605:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558619:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff989505542600) 00010000:00010000:2.0:1787784756.558622:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784756.558626:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558632:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc5f44 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558658:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558664:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558670:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558675:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784756.558678:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558685:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558693:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff989530cb1300) 00010000:00010000:2.0:1787784756.558695:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784756.558702:0:5813:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f2d expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558724:0:5813:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f2d expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558733:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f2d expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.558759:0:5813:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d200/0xb0ce63137acc5e79 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f2d expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784756.558772:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adf0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668395776:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784756.558848:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f540 going to sleep for 11 seconds 00000100:00100000:2.0:1787784756.563412:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.564991:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658117504 00000100:00100000:1.0:1787784756.565004:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658117504 00000100:00100000:3.0:1787784756.565259:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 204 00000100:00100000:3.0:1787784756.565272:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:27182:x1874627658117504:12345-192.168.203.43@tcp:49:diff.0 00000100:00100000:3.0:1787784756.565411:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:27182:x1874627658117504:12345-192.168.203.43@tcp:49:diff.0 Request processed in 142us (2007us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784756.565419:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 204 00000100:00100000:0.0:1787784756.566410:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668395776:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784756.566441:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566452:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5e72 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566485:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566499:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566511:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566520:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566527:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566534:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566540:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784756.566544:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566552:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566578:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff989530cb1300) 00010000:00010000:0.0:1787784756.566581:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784756.566585:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566591:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5f4b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566603:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566609:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566615:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566621:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784756.566623:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566629:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566637:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff989530cb1300) 00010000:00010000:0.0:1787784756.566639:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784756.566642:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.566648:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc5f3d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784756.566997:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:27183:x1874627658117248:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 19125us (20051us total) trans 4294967966 rc 0/0 00000100:00100000:0.0:1787784756.567007:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1031 00000020:00080000:0.0:1787784756.567012:0:5813:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:3.0:1787784756.569036:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784756.570232:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658117632 00000100:00100000:1.0:1787784756.570244:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658117632 00000100:00100000:1.0:1787784756.570312:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 275 00000100:00100000:1.0:1787784756.570323:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:27182:x1874627658117632:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:1.0:1787784756.570340:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784756.570362:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5f52 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f73 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784756.570389:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 158, mtime: 0x6a8f6e30, atime: 0x6a8f6e30, ctime: 0x6a8f6e30, blocks: 0xb 00010000:00010000:1.0:1787784756.570410:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=547s v=(0 0) 00010000:00010000:1.0:1787784756.570414:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784756.570424:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5f52 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f73 expref: 14 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.570440:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009ec1f683/0xb0ce63137acc5f52 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f73 expref: 14 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784756.570475:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c800, rc 0) 00000100:00100000:1.0:1787784756.570533:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:27182:x1874627658117632:12345-192.168.203.43@tcp:101:diff.0 Request processed in 212us (1504us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784756.570542:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 275 00000100:00080000:0.0:1787784756.570719:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:2.0:1787784756.571036:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784756.571041:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784756.571719:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:2.0:1787784756.571729:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784756.574322:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.574432:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658117888 00000100:00100000:0.0:1787784756.574442:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658117888 00000100:00100000:0.0:1787784756.574485:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 205 00000100:00100000:0.0:1787784756.574494:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d6850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:27183:x1874627658117888:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1787784756.575127:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1c:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1787784756.575266:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967967 00010000:00010000:0.0:1787784756.575310:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.575326:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.575337:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.575345:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.575351:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.575358:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784756.575385:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787784756.575477:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d6850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:27183:x1874627658117888:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 982us (1162us total) trans 4294967968 rc 0/0 00000100:00100000:0.0:1787784756.575486:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 205 00000020:00080000:0.0:1787784756.575491:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1787784756.575911:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784756.575919:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.575928:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784756.575932:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.575940:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.575952:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff98950a162200) 00010000:00010000:2.0:1787784756.575962:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787784756.575966:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.575973:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc5f59 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784756.575989:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784756.575991:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784756.575992:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784756.582247:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.582344:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658118016 00000100:00100000:0.0:1787784756.582354:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658118016 00002000:00100000:0.0:1787784756.582404:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x14:0x0] (0->4095) req@ffff98951b5bda40 x1874627658118016/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:637/0 lens 488/0 e 0 to 0 dl 1787784767 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784756.582415:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5bda40 with x1874627658118016 ext(0->4095) 00002000:00010000:0.0:1787784756.582420:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b5bda40 00000100:00100000:0.0:1787784756.582452:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 95 00000100:00100000:0.0:1787784756.582462:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658118016:12345-192.168.203.43@tcp:3:diff.0 00000100:00100000:0.0:1787784756.582700:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658118016:12345-192.168.203.43@tcp:3:diff.0 Request processed in 239us (463us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784756.582709:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 95 00002000:00100000:0.0:1787784756.582717:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x14:0x0] (0->4095) req@ffff98951b5bda40 x1874627658118016/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:637/0 lens 488/4536 e 0 to 0 dl 1787784767 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784756.582724:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5bda40 with x1874627658118016 ext(0->4095) 00002000:00010000:0.0:1787784756.582728:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98951b5bda40 00000100:00100000:3.0:1787784756.587981:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784756.588404:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658118272 00000100:00100000:0.0:1787784756.588414:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658118272 00000100:00100000:0.0:1787784756.588473:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 206 00000100:00100000:0.0:1787784756.588483:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:27182:x1874627658118272:12345-192.168.203.43@tcp:35:diff.0 00000100:00100000:0.0:1787784756.588738:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:27182:x1874627658118272:12345-192.168.203.43@tcp:35:diff.0 Request processed in 255us (764us total) trans 4294967969 rc 0/0 00000100:00100000:0.0:1787784756.588746:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 206 00000100:00100000:0.0:1787784756.724876:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668395904:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784756.724948:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784756.725028:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668395904 00000100:00100000:0.0:1787784756.725035:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668395904 00000100:00100000:0.0:1787784756.725078:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:0.0:1787784756.725087:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668395904:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784756.725109:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3857711104 avail 3854565376 00000100:00100000:0.0:1787784756.725172:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668395904:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784756.725180:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:0.0:1787784756.725232:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668395904:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784757.004220:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784757.004283:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784757.004321:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784757.004336:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658118400 00000100:00100000:0.0:1787784757.004350:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658118400 00000100:00100000:3.0:1787784757.004352:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784757.004394:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784757.004431:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:3.0:1787784757.004441:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784757.004451:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658118400:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784757.004495:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784757.004554:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658118400:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 111us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784757.004564:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:2.0:1787784757.004590:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658118912 00000100:00100000:2.0:1787784757.004595:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658118912 00000100:00100000:0.0:1787784757.004607:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658118656 00000100:00100000:0.0:1787784757.004611:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658118656 00000100:00100000:2.0:1787784757.004625:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658118528 00000100:00100000:2.0:1787784757.004628:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658118528 00000100:00100000:0.0:1787784757.004630:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119168 00000100:00100000:0.0:1787784757.004632:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119168 00000100:00100000:0.0:1787784757.004676:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658118784 00000100:00100000:0.0:1787784757.004679:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658118784 00000100:00100000:0.0:1787784757.004693:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119040 00000100:00100000:0.0:1787784757.004695:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119040 00000100:00100000:0.0:1787784757.004710:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 312 00000100:00100000:0.0:1787784757.004716:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2354:x1874627658118656:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784757.004816:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 363 00000100:00100000:2.0:1787784757.004823:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2354:x1874627658118912:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784757.006636:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 364 00000100:00100000:3.0:1787784757.006674:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:2354:x1874627658118528:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784757.006781:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:2354:x1874627658118528:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 115us (2386us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784757.006803:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 364 00000100:00100000:3.0:1787784757.006853:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 313 00000100:00100000:3.0:1787784757.006863:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658119168:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784757.010795:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2354:x1874627658118912:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 5882us (6385us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784757.010813:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 363 00000100:00080000:2.0:1787784757.010870:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784757.010877:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784757.011038:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2354:x1874627658118656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 6319us (6752us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784757.011049:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 312 00000100:00100000:0.0:1787784757.011281:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 314 00000100:00100000:0.0:1787784757.011297:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658118784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784757.012717:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:3.0:1787784757.012741:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658119040:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784757.013871:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658119040:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1131us (9368us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784757.013885:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:3.0:1787784757.014969:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658119168:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 8094us (10604us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784757.014984:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 313 00000100:00100000:0.0:1787784757.015481:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658118784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 4180us (11034us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784757.015496:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 314 00010000:00010000:0.0:1787784757.172306:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784757.307735:0:19737:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784757.307746:0:19737:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784757.314311:0:19737:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 28 count 5 00000040:00080000:1.0:1787784757.314322:0:19737:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784758.193593:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784758.771945:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668396032:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784758.772006:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784758.772073:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396032 00000100:00100000:2.0:1787784758.772080:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396032 00000100:00100000:2.0:1787784758.772125:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:2.0:1787784758.772132:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668396032:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784758.772187:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 78184448 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784758.772233:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668396032:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784758.772240:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00080000:2.0:1787784758.772258:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784758.772289:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668396032:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784758.772383:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668396288:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772407:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784758.772425:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784758.772430:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668396416:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772446:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784758.772473:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396288 00000100:00100000:2.0:1787784758.772476:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396288 00000100:00100000:2.0:1787784758.772496:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396416 00000100:00100000:2.0:1787784758.772498:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396416 00000100:00100000:2.0:1787784758.772516:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:2.0:1787784758.772521:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668396288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772551:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668396288:12345-0@lo:400:kworker.0 Request processed in 30us (143us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784758.772558:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:2.0:1787784758.772567:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1787784758.772572:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668396416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772592:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668396416:12345-0@lo:400:kworker.0 Request processed in 20us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784758.772597:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1787784758.772618:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668396160:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772642:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784758.772668:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668396672:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772685:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784758.772698:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784758.772702:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668396544:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772719:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784758.772739:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396160 00000100:00100000:2.0:1787784758.772742:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396160 00000100:00100000:2.0:1787784758.772763:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 550 00000100:00100000:2.0:1787784758.772768:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668396160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772810:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668396160:12345-0@lo:400:kworker.0 Request processed in 42us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784758.772816:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 550 00000100:00100000:2.0:1787784758.772837:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396544 00000100:00100000:2.0:1787784758.772841:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396544 00000100:00100000:2.0:1787784758.772861:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1787784758.772867:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668396544:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772890:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668396544:12345-0@lo:400:kworker.0 Request processed in 24us (170us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784758.772895:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00080000:2.0:1787784758.772912:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784758.772915:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784758.772931:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668396416:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772939:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668396288:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772962:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668396544:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772969:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668396160:0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.772990:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396672 00000100:00100000:2.0:1787784758.772992:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396672 00000100:00100000:2.0:1787784758.773012:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:2.0:1787784758.773017:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668396672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784758.773043:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668396672:12345-0@lo:400:kworker.0 Request processed in 26us (357us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784758.773049:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00080000:2.0:1787784758.773060:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784758.773071:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668396672:0@lo:400:kworker.0 00010000:00010000:2.0:1787784759.219496:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784760.241364:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784760.368119:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967355 is committed 00000001:00080000:0.0:1787784760.368134:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967356 is committed 00010000:00010000:2.0:1787784761.265295:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784761.841857:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784761.841889:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668396800:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784761.841989:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784761.842082:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396800 00000100:00100000:3.0:1787784761.842090:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396800 00000100:00100000:3.0:1787784761.842176:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:3.0:1787784761.842186:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668396800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784761.842228:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784761.842277:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668396800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784761.842309:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00080000:3.0:1787784761.842334:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784761.842677:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668396800:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1787784761.926957:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967969 is committed 00000001:00080000:2.0:1787784761.927010:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967968 is committed 00000020:00080000:2.0:1787784761.927036:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989518a329c0 refcount 2 committed 0 result 0 batchid 18 00000020:00080000:2.0:1787784761.927042:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989508cff800 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989518a32840 00000020:00080000:2.0:1787784761.927046:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989508cffa00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff98953f052400 00000020:00080000:2.0:1787784761.927076:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 18 committed batchid 17 00000020:00080000:2.0:1787784761.927330:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 17 00000001:00080000:1.0:1787784761.940340:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967349 is committed 00000001:00080000:1.0:1787784761.940373:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967350 is committed 00000100:00100000:0.0:1787784762.121410:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784762.121735:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784762.121811:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784762.121989:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119424 00000100:00100000:2.0:1787784762.121999:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119424 00000100:00080000:3.0:1787784762.122037:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784762.122067:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 368 00000100:00100000:2.0:1787784762.122076:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:2354:x1874627658119424:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784762.122169:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:2354:x1874627658119424:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 93us (887us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784762.122177:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 368 00000100:00100000:0.0:1787784762.125706:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784762.125739:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784762.126175:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119680 00000100:00100000:1.0:1787784762.126185:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119680 00000100:00100000:1.0:1787784762.126242:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119936 00000100:00100000:1.0:1787784762.126246:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119936 00000100:00100000:1.0:1787784762.126271:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 317 00000100:00100000:1.0:1787784762.126281:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658119680:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:3.0:1787784762.126417:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784762.126422:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784762.126597:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658119680:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 316us (4861us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784762.126606:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 317 00000100:00100000:1.0:1787784762.126625:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 318 00000100:00100000:1.0:1787784762.126633:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658119936:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784762.126671:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658119936:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (4861us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784762.126693:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 318 00000100:00100000:0.0:1787784762.126802:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784762.126899:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784762.127092:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119552 00000100:00100000:2.0:1787784762.127097:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119552 00000100:00100000:2.0:1787784762.127128:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120064 00000100:00100000:2.0:1787784762.127131:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120064 00000100:00100000:2.0:1787784762.127161:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 319 00000100:00100000:2.0:1787784762.127169:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2354:x1874627658119552:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784762.130158:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2354:x1874627658119552:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 2966us (4394us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784762.130175:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 319 00000100:00100000:2.0:1787784762.130215:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00100000:2.0:1787784762.130231:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658120064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:3.0:1787784762.130594:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784762.130599:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784762.131225:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119296 00000100:00100000:3.0:1787784762.131235:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119296 00000100:00100000:3.0:1787784762.131282:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 551 00000100:00100000:3.0:1787784762.131303:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658119296:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784762.131353:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658119296:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 64us (5656us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784762.131360:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 551 00000100:00100000:1.0:1787784762.132219:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658119808 00000100:00100000:1.0:1787784762.132228:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658119808 00000100:00100000:1.0:1787784762.132286:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 369 00000100:00080000:3.0:1787784762.132374:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784762.132409:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2354:x1874627658119808:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784762.132463:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2354:x1874627658119808:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 58us (5662us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784762.132471:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 369 00000100:00100000:2.0:1787784762.132938:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658120064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 2710us (6033us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784762.132948:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 320 00010000:00010000:2.0:1787784762.289512:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784762.566886:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784762.567005:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120192 00000100:00100000:2.0:1787784762.567015:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120192 00000100:00100000:2.0:1787784762.567076:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1032 00000100:00100000:2.0:1787784762.567087:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27154:x1874627658120192:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1787784762.567210:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+41:27154:x1874627658120192:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 124us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784762.567218:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1032 00000040:00080000:3.0:1787784762.740837:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784762.740850:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784762.741378:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 28 count 3 00000040:00080000:3.0:1787784762.741389:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784763.317103:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784763.894723:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668396928:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.894825:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784763.894888:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668397440:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.894937:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784763.894971:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784763.894978:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539100f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668397312:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.895006:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784763.895130:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668397056:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.895182:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784763.895204:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668397568:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.895226:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784763.895244:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784763.895250:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539101e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668397184:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.895347:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784763.895367:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539100b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668397696:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784763.895392:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784763.895434:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668396928 00000100:00100000:2.0:1787784763.895443:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668396928 00000100:00100000:2.0:1787784763.895500:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 552 00000100:00100000:2.0:1787784763.895509:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668396928:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.896204:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668396928:12345-0@lo:400:kworker.0 Request processed in 694us (1377us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784763.896214:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 552 00000100:00100000:2.0:1787784763.896342:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668397440 00000100:00100000:2.0:1787784763.896348:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668397440 00000100:00100000:2.0:1787784763.896400:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668397056 00000100:00100000:2.0:1787784763.896403:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668397056 00000100:00100000:2.0:1787784763.896417:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668397184 00000100:00100000:2.0:1787784763.896420:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668397184 00000100:00100000:2.0:1787784763.896436:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1787784763.896451:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668397440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.896489:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668397440:12345-0@lo:400:kworker.0 Request processed in 40us (1552us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784763.896496:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1787784763.896510:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1787784763.896516:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668397056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.896548:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668397056:12345-0@lo:400:kworker.0 Request processed in 32us (1365us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784763.896554:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1787784763.896566:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:2.0:1787784763.896572:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668397184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.896600:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668397184:12345-0@lo:400:kworker.0 Request processed in 28us (1253us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784763.896606:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:2.0:1787784763.896626:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668397312 00000100:00100000:2.0:1787784763.896631:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668397312 00000100:00100000:2.0:1787784763.896663:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668397568 00000100:00100000:2.0:1787784763.896666:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668397568 00000100:00100000:2.0:1787784763.896689:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 321 00000100:00100000:2.0:1787784763.896696:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668397312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.896730:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668397312:12345-0@lo:400:kworker.0 Request processed in 35us (1723us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784763.896748:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 321 00000100:00100000:2.0:1787784763.896760:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:2.0:1787784763.896766:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668397568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.896793:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668397568:12345-0@lo:400:kworker.0 Request processed in 28us (1567us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784763.896807:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:2.0:1787784763.896841:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668397696 00000100:00100000:2.0:1787784763.896846:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668397696 00000100:00100000:2.0:1787784763.896872:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:2.0:1787784763.896879:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668397696:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784763.896912:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 78184448 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784763.896946:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668397696:12345-0@lo:13:osp-pre-1-0.0 Request processed in 68us (1553us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784763.896951:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00080000:2.0:1787784763.896974:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784763.896979:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784763.896982:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784763.897029:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539101e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668397184:0@lo:400:kworker.0 00000100:00100000:0.0:1787784763.898246:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967350 gen 2 00000004:00080000:0.0:1787784763.898263:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967350 committed transno 4294967350 00000100:00100000:0.0:1787784763.898301:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539100f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668397312:0@lo:400:kworker.0 00000100:00100000:0.0:1787784763.898316:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668396928:0@lo:400:kworker.0 00000100:00100000:0.0:1787784763.898331:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668397440:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.899573:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967356 gen 2 00000004:00080000:2.0:1787784763.899586:0:3276:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967355 committed transno 4294967356 00000004:00080000:2.0:1787784763.899608:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402ade580, transno 4294967356 00000100:00100000:2.0:1787784763.899649:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539100b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668397696:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784763.899686:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668397056:0@lo:400:kworker.0 00000100:00100000:2.0:1787784763.899700:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668397568:0@lo:400:kworker.0 00000001:02000400:1.0:1787784764.283191:0:19937:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 18:52:42 (1787784762) 00010000:00010000:2.0:1787784764.351250:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784765.361349:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787784765.537588:0:19987:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 00010000:00010000:0.0:1787784766.385386:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784766.962196:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668397824:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784766.962304:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784766.971397:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668397824 00000100:00100000:1.0:1787784766.971417:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668397824 00000100:00100000:1.0:1787784766.971498:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1787784766.971509:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668397824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784766.971560:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784766.971671:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adfc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668397824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 165us (9367us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784766.971680:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00080000:1.0:1787784766.971752:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784766.972586:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668397824:0@lo:13:osp-pre-0-0.0 00000020:00080000:1.0:1787784767.146914:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 18 batchid updated 00000020:00080000:0.0:1787784767.147283:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 18 00000020:00080000:0.0:1787784767.147294:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 18 00000001:02000400:1.0:1787784767.164247:0:20036:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 18:52:45 (1787784765) 00000100:00100000:3.0:1787784767.239222:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.239262:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.239286:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.239314:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.239339:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.239369:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120704 00000100:00100000:1.0:1787784767.239373:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120448 00000100:00100000:1.0:1787784767.239384:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120448 00000100:00100000:3.0:1787784767.239385:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120704 00000100:00100000:1.0:1787784767.239452:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 373 00000100:00100000:1.0:1787784767.239463:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:2357:x1874627658120448:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784767.239544:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:2357:x1874627658120448:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 83us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784767.239552:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 373 00000100:00080000:1.0:1787784767.239577:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784767.239583:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120960 00000100:00080000:1.0:1787784767.239583:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784767.239589:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120960 00000100:00100000:3.0:1787784767.239614:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120576 00000100:00100000:3.0:1787784767.239618:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120576 00000100:00100000:3.0:1787784767.239633:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 323 00000100:00100000:3.0:1787784767.239640:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658120704:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784767.239669:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658120704:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.239677:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 323 00000100:00100000:3.0:1787784767.239689:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:3.0:1787784767.239696:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658120960:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784767.240858:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658120960:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1157us (1565us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.240868:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:3.0:1787784767.240891:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:3.0:1787784767.240899:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658120576:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784767.241021:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.241088:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120832 00000100:00100000:2.0:1787784767.241102:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120832 00000100:00100000:2.0:1787784767.241180:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:2.0:1787784767.241190:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2357:x1874627658120832:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784767.241204:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.241228:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2357:x1874627658120832:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (207us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.241235:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:2.0:1787784767.242549:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658121088 00000100:00100000:2.0:1787784767.242558:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658121088 00000100:00100000:2.0:1787784767.242598:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 326 00000100:00100000:2.0:1787784767.242606:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658121088:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784767.242683:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658121088:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (1480us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.242699:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 326 00000100:00100000:2.0:1787784767.242739:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658120320 00000100:00100000:2.0:1787784767.242743:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658120320 00000100:00100000:2.0:1787784767.242770:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 553 00000100:00100000:2.0:1787784767.242786:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658120320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784767.242822:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658120320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 45us (3507us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.242829:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 553 00000100:00100000:3.0:1787784767.243557:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658120576:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 2655us (4213us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.243567:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00080000:3.0:1787784767.243680:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784767.243686:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784767.243688:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784767.399732:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.400736:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658121216 00000100:00100000:2.0:1787784767.400748:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658121216 00000100:00100000:2.0:1787784767.400823:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1033 00000100:00100000:2.0:1787784767.400833:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:28083:x1874627658121216:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1787784767.400954:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:28083:x1874627658121216:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 123us (1240us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.400962:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1033 00010000:00010000:2.0:1787784767.409622:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784767.425364:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.425616:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658121344 00000100:00100000:2.0:1787784767.425629:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658121344 00000100:00100000:2.0:1787784767.425794:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1034 00000100:00100000:2.0:1787784767.425807:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:28084:x1874627658121344:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784767.425838:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784767.428877:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f60 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f7a expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.429415:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.429436:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.429450:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.431351:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.431374:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.431386:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432648:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432751:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432759:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432765:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784767.432769:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432778:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432794:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784767.432797:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784767.432801:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432807:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f67 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432821:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432828:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432834:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432840:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784767.432947:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432955:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432965:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff98951c407900) 00010000:00010000:1.0:1787784767.432967:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784767.432971:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.432977:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f6e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.433023:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f60 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f7a expref: 90 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.433110:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784767.433113:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784767.433117:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f60 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f7a expref: 90 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.433137:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784767.433620:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:28084:x1874627658121344:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7813us (8267us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784767.433631:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1034 00000100:00100000:3.0:1787784767.439633:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.440488:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658121600 00000100:00100000:2.0:1787784767.440506:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658121600 00000100:00100000:2.0:1787784767.440645:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1035 00000100:00100000:2.0:1787784767.440657:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:28084:x1874627658121600:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:2.0:1787784767.440749:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784767.440788:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc5cb9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0e8c expref: 89 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.440814:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:2.0:1787784767.440820:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784767.441248:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.441267:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 40 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.441276:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.441335:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 40 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.441354:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 40 pid: 10850 timeout: 658 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784767.441371:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668397952:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784767.441500:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd43c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784767.445045:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668397952:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784767.445098:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784767.445436:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.445518:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658121728 00000100:00100000:2.0:1787784767.445527:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658121728 00010000:00010000:2.0:1787784767.445573:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 41 pid: 10850 timeout: 658 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784767.445615:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 314 00000100:00100000:2.0:1787784767.445625:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2357:x1874627658121728:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784767.445643:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784767.445651:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 41 pid: 10850 timeout: 658 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.445665:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 41 pid: 10850 timeout: 658 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.445727:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc5cf8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ea1 expref: 41 pid: 10850 timeout: 658 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.445748:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:2.0:1787784767.445754:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.445764:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.445772:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.445806:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784767.445874:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+40:2357:x1874627658121728:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 249us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.445882:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 314 00010000:00010000:1.0:1787784767.445904:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.445923:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.445969:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.445982:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.445994:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784767.450724:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 512 00010000:00010000:0.0:1787784767.450743:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.450754:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.450761:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.450767:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784767.450771:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.450779:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.450795:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784767.450798:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784767.450802:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.450809:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5f75 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784767.450843:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000096306e73 reply state 000000009d1ca9b0 transno 4294967970 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784767.451705:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784767.451829:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:28084:x1874627658121600:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 11169us (13110us total) trans 4294967970 rc 0/0 00000100:00100000:0.0:1787784767.451839:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1035 00010000:00010000:2.0:1787784767.454841:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.454869:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.454878:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.454884:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.454889:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.454926:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.454963:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff98951c407900) 00010000:00010000:2.0:1787784767.454967:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.454972:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.454983:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc5f7c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784767.464616:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784767.465206:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658121856 00000100:00100000:1.0:1787784767.465217:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658121856 00000100:00100000:1.0:1787784767.465289:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1036 00000100:00100000:1.0:1787784767.465300:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+89:28085:x1874627658121856:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784767.465325:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784767.465347:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f83 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f81 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.465414:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f8a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.465428:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f8a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.465450:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f8a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784767.465508:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784767.465513:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 27 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784767.465538:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f8a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.465575:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc5f83 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f81 expref: 91 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.465593:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f8a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f81 expref: 90 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.465601:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407500) 00010000:00010000:1.0:1787784767.465604:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784767.465608:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784767.465670:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:28085:x1874627658121856:12345-192.168.203.43@tcp:101:bash.0 Request processed in 372us (1063us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784767.465677:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1036 00000100:00100000:3.0:1787784767.470624:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.471260:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658121984 00000100:00100000:0.0:1787784767.471285:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658121984 00000100:00100000:0.0:1787784767.471644:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1037 00000100:00100000:0.0:1787784767.471657:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+90:28085:x1874627658121984:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784767.471677:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784767.471698:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f91 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f88 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471772:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471787:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471798:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471811:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471818:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471839:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471965:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5fa6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471974:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5fa6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.471981:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5fa6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784767.472009:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 512 00000004:00200000:0.0:1787784767.472013:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784767.472251:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5fa6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472275:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472283:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472289:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472296:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784767.472300:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472308:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472330:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407500) 00010000:00010000:0.0:1787784767.472333:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784767.472337:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472343:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5f98 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472357:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472363:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472368:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472373:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784767.472376:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472382:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472390:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff989503a6da00) 00010000:00010000:0.0:1787784767.472392:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784767.472394:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472400:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5f9f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472443:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5f91 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f88 expref: 92 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472457:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030c00/0xb0ce63137acc5fa6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f88 expref: 91 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.472464:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784767.472466:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784767.472469:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030c00, rc 0) 00000100:00100000:0.0:1787784767.472638:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28085:x1874627658121984:12345-192.168.203.43@tcp:101:bash.0 Request processed in 983us (2029us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784767.472648:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1037 00000100:00100000:1.0:1787784767.478858:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.478971:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658122112 00000100:00100000:2.0:1787784767.478984:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658122112 00000100:00100000:2.0:1787784767.479126:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1038 00000100:00100000:2.0:1787784767.479139:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28085:x1874627658122112:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784767.479189:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.479214:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5fad lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f8f expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479296:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479310:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479322:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479337:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479359:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479367:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479435:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479442:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479447:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479453:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.479457:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479464:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479478:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.479481:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.479486:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479493:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fb4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479505:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479518:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479524:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479529:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.479532:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479537:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479545:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989519d44700) 00010000:00010000:2.0:1787784767.479548:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.479550:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479556:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fbb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479591:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5fad lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f8f expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479603:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.479605:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.479610:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5fad lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f8f expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.479633:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784767.479700:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28085:x1874627658122112:12345-192.168.203.43@tcp:101:bash.0 Request processed in 564us (857us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784767.479708:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1038 00000100:00100000:1.0:1787784767.499059:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.499718:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658122240 00000100:00100000:2.0:1787784767.499733:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658122240 00000100:00100000:2.0:1787784767.499871:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1039 00000100:00100000:2.0:1787784767.499883:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+40:28086:x1874627658122240:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784767.499906:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.499926:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5fc2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f96 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.499986:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.500000:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.500013:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784767.500081:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784767.500085:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 27 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784767.500106:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.500129:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5fc2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 42 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.500200:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 41 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.500210:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407500) 00010000:00010000:2.0:1787784767.500213:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784767.500216:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784767.500492:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+41:28086:x1874627658122240:12345-192.168.203.43@tcp:101:rm.0 Request processed in 610us (1447us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.500500:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1039 00000100:00100000:3.0:1787784767.505809:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.506419:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658122368 00000100:00100000:2.0:1787784767.506437:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658122368 00000100:00100000:2.0:1787784767.506503:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1040 00000100:00100000:2.0:1787784767.506514:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:28086:x1874627658122368:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784767.506534:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.506554:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5fd0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0f9d expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506618:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506633:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506645:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506659:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506680:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506687:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506807:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5fe5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506816:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5fe5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506823:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5fe5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784767.506853:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 512 00000004:00200000:2.0:1787784767.506856:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784767.506877:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5fe5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506889:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506896:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506902:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506907:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.506911:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506925:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506937:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407500) 00010000:00010000:2.0:1787784767.506940:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.506945:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506951:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc5fd7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506963:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506969:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506974:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506979:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.506981:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506987:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.506993:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff989519d44700) 00010000:00010000:2.0:1787784767.506996:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.506998:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.507012:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc5fde lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.507043:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc5fd0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f9d expref: 43 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.507056:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc000/0xb0ce63137acc5fe5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f9d expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.507063:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.507065:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784767.507067:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc000, rc 0) 00000100:00100000:2.0:1787784767.507242:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28086:x1874627658122368:12345-192.168.203.43@tcp:101:rm.0 Request processed in 728us (1447us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.507250:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1040 00000100:00100000:1.0:1787784767.510618:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.511405:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658122496 00000100:00100000:0.0:1787784767.511422:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658122496 00000100:00100000:0.0:1787784767.511565:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1041 00000100:00100000:0.0:1787784767.511577:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28086:x1874627658122496:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784767.511605:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784767.511633:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5fec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fa4 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.511838:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.511857:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.511872:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.511908:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.511916:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.511924:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512009:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512017:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512024:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512031:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784767.512036:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512044:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512058:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784767.512062:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784767.512138:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512168:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc5ff3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512197:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512203:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512209:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512214:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784767.512217:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512225:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512234:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989503a6da00) 00010000:00010000:0.0:1787784767.512236:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784767.512238:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512244:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc5ffa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512286:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5fec lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fa4 expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512364:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784767.512367:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784767.512377:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc5fec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fa4 expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.512388:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031800, rc 0) 00000100:00100000:0.0:1787784767.512610:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28086:x1874627658122496:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1034us (2004us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784767.512619:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1041 00000100:00100000:1.0:1787784767.516804:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.516906:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658122624 00000100:00100000:2.0:1787784767.516921:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658122624 00000100:00100000:2.0:1787784767.517006:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1042 00000100:00100000:2.0:1787784767.517017:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28086:x1874627658122624:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784767.517140:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.517194:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6001 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fab expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517427:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517442:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517466:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517482:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517488:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517494:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517557:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517563:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517570:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517575:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.517578:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517584:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517595:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.517598:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.517602:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517616:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6008 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517630:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517636:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517642:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517648:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.517651:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517658:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517665:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989519d44700) 00010000:00010000:2.0:1787784767.517668:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.517671:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517678:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc600f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517720:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6001 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fab expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517744:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.517746:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.517750:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6001 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fab expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.517760:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784767.517829:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28086:x1874627658122624:12345-192.168.203.43@tcp:101:rm.0 Request processed in 813us (1038us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784767.517837:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1042 00000100:00100000:3.0:1787784767.535721:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.535826:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658122752 00000100:00100000:2.0:1787784767.535843:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658122752 00000100:00100000:2.0:1787784767.535922:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1043 00000100:00100000:2.0:1787784767.535933:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28087:x1874627658122752:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784767.535952:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.535974:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6016 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fb2 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536198:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536212:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536223:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536237:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536259:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536266:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536334:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536342:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536348:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536354:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.536358:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536366:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536378:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.536381:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.536385:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536390:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc601d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536404:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536416:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536422:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536428:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.536431:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536438:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536446:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989519d44700) 00010000:00010000:2.0:1787784767.536449:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.536451:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536458:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6024 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536490:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6016 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fb2 expref: 43 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536500:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.536502:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.536505:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6016 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fb2 expref: 43 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.536526:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784767.536584:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28087:x1874627658122752:12345-192.168.203.43@tcp:101:cp.0 Request processed in 653us (888us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784767.536592:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1043 00000100:00100000:3.0:1787784767.540957:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.541227:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658122880 00000100:00100000:2.0:1787784767.541240:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658122880 00000100:00100000:2.0:1787784767.541325:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1044 00000100:00100000:2.0:1787784767.541337:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28087:x1874627658122880:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784767.541366:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.541395:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc602b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fb9 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541692:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541726:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541741:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541757:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541764:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541771:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541865:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541873:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541879:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541885:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.541889:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541896:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541911:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.541914:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.541919:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541926:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6032 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541942:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541947:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541952:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541963:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.541966:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541972:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541979:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989519d44600) 00010000:00010000:2.0:1787784767.541981:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.541984:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.541989:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6039 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.542031:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc602b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fb9 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.542041:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.542044:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.542047:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc602b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fb9 expref: 43 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.542056:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784767.542744:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28087:x1874627658122880:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1405us (1796us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784767.542756:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1044 00000100:00100000:2.0:1787784767.546628:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.546748:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658123136 00000100:00100000:0.0:1787784767.546761:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658123136 00000100:00100000:0.0:1787784767.546841:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1045 00000100:00100000:0.0:1787784767.546854:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28087:x1874627658123136:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784767.546885:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784767.546913:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6040 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fc0 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.547095:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.547112:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.547125:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.547141:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.547175:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.547182:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784767.548000:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x17:0x0], index 1: 0 00010000:00010000:0.0:1787784767.548246:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548260:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548270:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548313:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548321:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548328:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548334:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784767.548339:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548347:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548362:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:0.0:1787784767.548365:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784767.548370:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548389:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc6047 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784767.548405:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c32c8c4a reply state 00000000ecb18988 transno 4294967971 ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784767.548414:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784767.548456:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6040 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 45 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548474:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.548483:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:0.0:1787784767.548485:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784767.548488:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:0.0:1787784767.550503:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28087:x1874627658123136:12345-192.168.203.43@tcp:101:cp.0 Request processed in 3637us (3873us total) trans 4294967971 rc 0/0 00000100:00100000:0.0:1787784767.550518:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1045 00000100:00100000:2.0:1787784767.564353:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784767.565011:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.565030:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.565040:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.565047:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784767.565053:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.565278:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.565309:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989503a6da00) 00010000:00010000:1.0:1787784767.565313:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784767.565319:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.565327:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc604e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784767.566059:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658123264 00000100:00100000:3.0:1787784767.566075:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658123264 00000100:00100000:3.0:1787784767.566176:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 276 00000100:00100000:3.0:1787784767.566188:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658123264:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784767.566211:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784767.566234:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fc7 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784767.566264:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x17:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1787784767.566288:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=558s v=(0 0) 00010000:00010000:3.0:1787784767.566293:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784767.566304:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.566320:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.566333:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784767.566414:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658123264:12345-192.168.203.43@tcp:101:cp.0 Request processed in 227us (2075us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.566423:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 276 00000100:00100000:1.0:1787784767.578044:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.578761:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658123392 00000100:00100000:2.0:1787784767.578775:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658123392 00000100:00100000:2.0:1787784767.578904:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 277 00000100:00100000:2.0:1787784767.578917:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:28087:x1874627658123392:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784767.578942:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.578968:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fce expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.579008:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.579019:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784767.579025:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784767.579030:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.579242:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.579259:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 12 pid: 6669 timeout: 658 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784767.579276:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668398080:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784767.579582:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd43c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784767.584805:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.585005:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658123520 00000100:00100000:0.0:1787784767.585017:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658123520 00010000:00010000:0.0:1787784767.585081:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 13 pid: 6669 timeout: 658 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784767.585136:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:0.0:1787784767.585166:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658123520:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784767.585197:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784767.585245:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 13 pid: 6669 timeout: 658 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.585261:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 13 pid: 6669 timeout: 658 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.585304:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784767.585310:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784767.585320:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.585333:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.585381:0:11303:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784767.585399:0:11303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11303:1874627668398208:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784767.585877:0:11303:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f780 going to sleep for 11 seconds 00010000:00010000:2.0:1787784767.588904:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc6063 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc7 expref: 13 pid: 6669 timeout: 658 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784767.588969:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668398080:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784767.589012:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.589030:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784767.589261:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:28087:x1874627658123392:12345-192.168.203.43@tcp:101:cp.0 Request processed in 10345us (11230us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.589272:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 277 00000100:00100000:2.0:1787784767.592421:0:11303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11303:1874627668398208:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784767.592464:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784767.592566:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658123520:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 7419us (7775us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.592580:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:1.0:1787784767.595676:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.595861:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658123776 00000100:00100000:0.0:1787784767.595873:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658123776 00000100:00100000:0.0:1787784767.595998:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 207 00000100:00100000:0.0:1787784767.596010:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28087:x1874627658123776:12345-192.168.203.43@tcp:35:cp.0 00000001:00080000:0.0:1787784767.596440:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967972 00000100:00100000:0.0:1787784767.596800:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28087:x1874627658123776:12345-192.168.203.43@tcp:35:cp.0 Request processed in 789us (1138us total) trans 4294967973 rc 0/0 00000100:00100000:0.0:1787784767.596831:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 207 00000020:00080000:0.0:1787784767.596837:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1787784767.632209:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784767.632383:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658123904 00000100:00100000:1.0:1787784767.632396:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658123904 00000100:00100000:1.0:1787784767.632478:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1046 00000100:00100000:1.0:1787784767.632490:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28090:x1874627658123904:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784767.632517:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784767.632544:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6071 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fd5 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.632663:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.632684:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.632698:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.634701:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.634795:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.634811:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.634954:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.634963:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.634989:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784767.635044:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00000004:00200000:2.0:1787784767.635050:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784767.635922:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.635947:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.635959:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.635966:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.635974:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.635981:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.635986:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.635996:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636014:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784767.636018:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.636023:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636049:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6078 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636532:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636639:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636647:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636652:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784767.636657:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636664:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636795:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989519d44600) 00010000:00010000:2.0:1787784767.636799:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784767.636804:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636811:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc607f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636877:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6071 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 93 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636911:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.636919:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:2.0:1787784767.636921:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784767.636924:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784767.637036:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28090:x1874627658123904:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4546us (4839us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.637048:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1046 00000100:00100000:2.0:1787784767.641232:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.641336:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658124032 00000100:00100000:3.0:1787784767.641349:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658124032 00000100:00100000:3.0:1787784767.641442:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 278 00000100:00100000:3.0:1787784767.641454:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658124032:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784767.641479:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784767.641504:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ac822b5/0xb0ce63137acc608d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fdc expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784767.641584:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668398336:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784767.641687:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6fa80 going to sleep for 11 seconds 00002000:00010000:3.0:1787784767.645672:0:10688:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb size: 0 -> 1516 00002000:00010000:3.0:1787784767.645686:0:10688:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb mtime: 0 -> 1787784767 00002000:00010000:3.0:1787784767.645689:0:10688:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb atime: 0 -> 1787784766 00002000:00010000:3.0:1787784767.645691:0:10688:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb ctime: 0 -> 1787784767 00000100:00100000:1.0:1787784767.645886:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668398336:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784767.645938:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ac822b5/0xb0ce63137acc608d lrc: 1/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fdc expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.645970:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ac822b5/0xb0ce63137acc608d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fdc expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784767.646300:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:1.0:1787784767.646444:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658124032:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4989us (5225us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784767.646456:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 278 00000100:00100000:0.0:1787784767.655780:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.655975:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658124160 00000100:00100000:3.0:1787784767.655988:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658124160 00000100:00100000:3.0:1787784767.656062:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1047 00000100:00100000:3.0:1787784767.656074:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28090:x1874627658124160:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784767.656103:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784767.656131:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6094 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fe3 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.656248:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.656265:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.656281:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784767.656386:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 512 00000004:00200000:3.0:1787784767.656391:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784767.656419:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.656453:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6094 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 94 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.656476:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 93 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.656485:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:3.0:1787784767.656488:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784767.656492:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784767.656574:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28090:x1874627658124160:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 502us (808us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.656583:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1047 00000100:00100000:2.0:1787784767.661350:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.661473:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658124288 00000100:00100000:0.0:1787784767.661486:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658124288 00000100:00100000:0.0:1787784767.661557:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 279 00000100:00100000:0.0:1787784767.661569:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658124288:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784767.661594:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784767.661620:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc60a2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fea expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784767.661695:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668398464:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784767.661815:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784767.667022:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668398464:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784767.667075:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc60a2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fea expref: 13 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.667104:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc60a2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fea expref: 13 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.667134:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031000, rc 0) 00000100:00100000:0.0:1787784767.667294:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658124288:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 5726us (5958us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784767.667305:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 279 00000100:00100000:1.0:1787784767.677272:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.677400:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658124544 00000100:00100000:2.0:1787784767.677411:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658124544 00000100:00100000:2.0:1787784767.677489:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1048 00000100:00100000:2.0:1787784767.677500:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28090:x1874627658124544:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:2.0:1787784767.677889:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].28, 136 off12000 00000100:00100000:2.0:1787784767.678196:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28090:x1874627658124544:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 696us (935us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.678206:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1048 00000040:00080000:3.0:1787784767.863670:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784767.863683:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784767.863739:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 29 count 4 00000040:00080000:3.0:1787784767.863747:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784767.864072:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514f00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668398592:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784767.864237:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784767.875225:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.875386:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658124672 00000100:00100000:2.0:1787784767.875397:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658124672 00000100:00100000:2.0:1787784767.875470:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1049 00000100:00100000:2.0:1787784767.875483:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28100:x1874627658124672:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784767.875531:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784767.875666:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28100:x1874627658124672:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 185us (456us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.875687:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1049 00000100:00100000:3.0:1787784767.876111:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514f00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668398592:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784767.893973:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.896746:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658124928 00000100:00100000:0.0:1787784767.896772:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658124928 00000100:00100000:0.0:1787784767.896861:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1050 00000100:00100000:0.0:1787784767.896886:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28100:x1874627658124928:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784767.896914:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784767.896941:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc60a9 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ff1 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.897086:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc60b0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.897106:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc60b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.897132:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc60b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784767.897166:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:0.0:1787784767.897345:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc60a9 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff1 expref: 49 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.897366:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc60b0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff1 expref: 48 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.897376:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:0.0:1787784767.897380:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784767.897384:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031800, rc 0) 00000100:00100000:0.0:1787784767.897486:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28100:x1874627658124928:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 603us (3526us total) trans 4294967974 rc 0/0 00000100:00100000:0.0:1787784767.897510:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1050 00000100:00100000:3.0:1787784767.909234:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.909284:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658125056 00000100:00100000:3.0:1787784767.909295:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658125056 00000100:00100000:3.0:1787784767.909351:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00100000:3.0:1787784767.909361:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658125056:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784767.909415:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.909438:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.909449:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784767.909454:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784767.909459:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 4/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.909497:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.909512:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 11 pid: 13825 timeout: 658 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784767.909529:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305161c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668398720:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784767.909616:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784767.911833:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.911961:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658125184 00000100:00100000:0.0:1787784767.911974:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658125184 00000100:00100000:0.0:1787784767.912054:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 45 00000100:00100000:0.0:1787784767.912066:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2357:x1874627658125184:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784767.912086:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784767.912119:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc60b0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff1 expref: 48 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.912181:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:0.0:1787784767.912186:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784767.912248:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658125184:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 185us (427us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784767.912256:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 45 00000100:00100000:3.0:1787784767.916887:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305161c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668398720:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784767.916927:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784767.923101:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.923241:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658125312 00000100:00100000:2.0:1787784767.923254:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658125312 00002000:00100000:2.0:1787784767.923331:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x17:0x0] (0->1515) req@ffff9895305161c0 x1874627658125312/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:648/0 lens 2008/0 e 0 to 0 dl 1787784778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784767.923343:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895305161c0 with x1874627658125312 ext(0->1515) 00010000:00010000:2.0:1787784767.923354:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 12 pid: 13825 timeout: 658 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.923373:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 12 pid: 13825 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784767.923381:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9895305161c0 00000100:00100000:2.0:1787784767.923418:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 11 00000100:00100000:2.0:1787784767.923430:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305161c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658125312:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784767.924287:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305161c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658125312:12345-192.168.203.43@tcp:4:cp.0 Request processed in 852us (1192us total) trans 4294967357 rc 0/0 00000100:00100000:2.0:1787784767.924298:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 11 00002000:00100000:2.0:1787784767.924308:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x17:0x0] (0->1515) req@ffff9895305161c0 x1874627658125312/t4294967357(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:648/0 lens 2008/448 e 0 to 0 dl 1787784778 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784767.924317:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895305161c0 with x1874627658125312 ext(0->1515) 00010000:00010000:2.0:1787784767.924324:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 14 pid: 13825 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.924353:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 14 pid: 13825 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784767.924360:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9895305161c0 00000100:00100000:0.0:1787784767.937759:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.938117:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658125440 00000100:00100000:3.0:1787784767.938130:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658125440 00010000:00010000:3.0:1787784767.938218:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 14 pid: 13825 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784767.938280:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 316 00000100:00100000:3.0:1787784767.938302:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658125440:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784767.938321:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784767.938350:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 14 pid: 13825 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.938366:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 14 pid: 13825 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.938398:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc606a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fce expref: 14 pid: 13825 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.938417:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=1s v=(1 0) 00010000:00010000:3.0:1787784767.938422:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784767.938428:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.938440:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.938493:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784767.938892:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658125440:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 600us (1147us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.938901:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 316 00010000:00010000:3.0:1787784767.939819:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939839:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939875:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939885:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939893:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939900:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784767.939905:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939915:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939951:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.939959:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60be lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784767.940046:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658125056:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 30682us (30822us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.940056:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00100000:0.0:1787784767.946475:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.947683:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658125696 00000100:00100000:3.0:1787784767.947716:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658125696 00000100:00100000:3.0:1787784767.947795:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1051 00000100:00100000:3.0:1787784767.947808:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28100:x1874627658125696:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784767.947968:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28100:x1874627658125696:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 162us (1508us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.947976:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1051 00000100:00100000:1.0:1787784767.956753:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.958493:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658125824 00000100:00100000:2.0:1787784767.958514:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658125824 00000100:00100000:2.0:1787784767.958744:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 281 00000100:00100000:2.0:1787784767.958757:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658125824:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784767.958782:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784767.958809:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0ff8 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784767.958851:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x17:0x0] initial LVB size: 1516, mtime: 0x6a8f6e3f, atime: 0x0, ctime: 0x6a8f6e3f, blocks: 0x1 00010000:00010000:2.0:1787784767.959044:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=559s v=(0 0) 00010000:00010000:2.0:1787784767.959051:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784767.959068:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.959092:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 13 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.959108:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784767.959208:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658125824:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 453us (2469us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.959235:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 281 00000100:00100000:1.0:1787784767.963639:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.964829:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658126080 00000100:00100000:0.0:1787784767.964844:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658126080 00000100:00100000:2.0:1787784767.965036:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1052 00000100:00100000:2.0:1787784767.965048:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28100:x1874627658126080:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784767.965217:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28100:x1874627658126080:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 170us (1591us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.965225:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1052 00000100:00100000:0.0:1787784767.970043:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.970259:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658126208 00000100:00100000:2.0:1787784767.970274:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658126208 00002000:00100000:2.0:1787784767.970351:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x17:0x0] (0->4095) req@ffff989539100f00 x1874627658126208/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:648/0 lens 488/0 e 0 to 0 dl 1787784778 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784767.970365:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539100f00 with x1874627658126208 ext(0->4095) 00002000:00010000:2.0:1787784767.970371:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539100f00 00000100:00100000:2.0:1787784767.970412:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 96 00000100:00100000:2.0:1787784767.970423:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658126208:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784767.971001:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658126208:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 577us (969us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784767.971012:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 96 00002000:00100000:2.0:1787784767.971021:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x17:0x0] (0->4095) req@ffff989539100f00 x1874627658126208/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:648/0 lens 488/4536 e 0 to 0 dl 1787784778 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784767.971029:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539100f00 with x1874627658126208 ext(0->4095) 00002000:00010000:2.0:1787784767.971033:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539100f00 00000100:00100000:1.0:1787784767.974904:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784767.975084:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658126336 00000100:00100000:3.0:1787784767.975096:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658126336 00000100:00100000:3.0:1787784767.975230:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1053 00000100:00100000:3.0:1787784767.975243:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28100:x1874627658126336:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784767.975267:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784767.975294:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc60cc lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b0fff expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.975378:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.975395:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.975411:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.975530:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc60cc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.975562:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784767.975572:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:3.0:1787784767.975575:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784767.975579:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6800, rc 0) 00000100:00100000:3.0:1787784767.975649:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28100:x1874627658126336:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 410us (760us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784767.975657:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1053 00000100:00100000:1.0:1787784767.980685:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.980835:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658126464 00000100:00100000:0.0:1787784767.980848:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658126464 00000100:00100000:0.0:1787784767.980981:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1054 00000100:00100000:0.0:1787784767.980995:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28100:x1874627658126464:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784767.981077:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784767.981261:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28100:x1874627658126464:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 268us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784767.981279:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1054 00000100:00100000:0.0:1787784767.984510:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784767.984887:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658126592 00000100:00100000:2.0:1787784767.984900:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658126592 00000100:00100000:2.0:1787784767.985029:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 282 00000100:00100000:2.0:1787784767.985040:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658126592:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784767.985100:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 3/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985123:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784767.985128:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784767.985135:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985197:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985238:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985248:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985256:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985264:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784767.985269:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985279:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985304:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784767.985313:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d0e0def3/0xb0ce63137acc60da lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784767.985410:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658126592:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 372us (914us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784767.985419:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 282 00000100:00100000:1.0:1787784767.995119:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784767.995392:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658126848 00000100:00100000:0.0:1787784767.995404:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658126848 00000100:00100000:0.0:1787784767.995478:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1055 00000100:00100000:0.0:1787784767.995490:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28100:x1874627658126848:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784767.995597:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.995619:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.995629:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.995672:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784767.995756:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 48 pid: 5811 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784767.995774:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668398848:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784767.996066:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f300 going to sleep for 11 seconds 00000100:00100000:1.0:1787784768.006611:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784768.006834:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658126976 00000100:00100000:3.0:1787784768.006847:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658126976 00010000:00010000:3.0:1787784768.006912:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 49 pid: 5811 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784768.006964:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 317 00000100:00100000:3.0:1787784768.006975:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658126976:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784768.007016:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784768.007025:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 49 pid: 5811 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.007041:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 49 pid: 5811 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.007069:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:3.0:1787784768.007075:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.007087:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.007096:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.007119:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784768.007231:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658126976:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 257us (633us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784768.007239:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 317 00010000:00010000:0.0:1787784768.008739:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc60d3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fff expref: 48 pid: 5811 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784768.008772:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668398848:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784768.008821:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.008834:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.008841:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.012536:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.012561:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.012569:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.012592:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784768.012597:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.012607:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.012627:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:1.0:1787784768.012631:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784768.012637:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.012644:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc60e1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784768.012919:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28100:x1874627658126848:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 17425us (17807us total) trans 4294967975 rc 0/0 00000100:00100000:1.0:1787784768.012933:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1055 00000040:00080000:2.0:1787784768.014244:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784768.014254:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784768.017202:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:2.0:1787784768.017231:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784768.020408:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784768.020521:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658127232 00000100:00100000:0.0:1787784768.020536:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658127232 00000100:00100000:0.0:1787784768.020827:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 208 00000100:00100000:0.0:1787784768.020842:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28100:x1874627658127232:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784768.021215:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28100:x1874627658127232:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 374us (819us total) trans 4294967976 rc 0/0 00000100:00100000:0.0:1787784768.021231:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 208 00010000:00010000:1.0:1787784768.433983:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784768.470742:0:20085:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784768.470750:0:20085:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784768.470797:0:20085:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 29 count 4 00000040:00080000:1.0:1787784768.470803:0:20085:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784768.533338:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784768.533387:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658127360 00000100:00100000:3.0:1787784768.533398:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658127360 00000100:00100000:3.0:1787784768.533457:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 283 00000100:00100000:3.0:1787784768.533467:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658127360:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784768.533484:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784768.533501:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1006 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.533539:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784768.533543:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784768.533553:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.533570:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.533584:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784768.533641:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2355:x1874627658127360:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 174us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784768.533648:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 283 00000100:00100000:1.0:1787784768.548892:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784768.548990:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658127616 00000100:00100000:3.0:1787784768.549000:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658127616 00000100:00100000:3.0:1787784768.549061:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1056 00000100:00100000:3.0:1787784768.549072:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28101:x1874627658127616:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784768.549091:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784768.549112:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc60ef lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b100d expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.549291:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc60f6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.549305:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc60f6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.549327:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc60f6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784768.549340:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:3.0:1787784768.549457:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc60ef lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b100d expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.549473:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc60f6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b100d expref: 95 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.549480:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:3.0:1787784768.549483:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1787784768.549486:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6e00, rc 0) 00000100:00100000:3.0:1787784768.549544:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:28101:x1874627658127616:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 474us (659us total) trans 4294967977 rc 0/0 00000100:00100000:3.0:1787784768.549552:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1056 00000100:00100000:0.0:1787784768.564299:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784768.564409:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658127744 00000100:00100000:3.0:1787784768.564419:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658127744 00000100:00100000:3.0:1787784768.564465:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00100000:3.0:1787784768.564475:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2355:x1874627658127744:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:3.0:1787784768.564517:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.564537:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.564627:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.564636:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784768.564640:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784768.564644:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.564678:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.564693:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 13 pid: 13825 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784768.564719:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530516580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668398976:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784768.564820:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.564830:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 12 pid: 10688 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784768.564839:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668399104:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784768.564862:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507b69540 going to sleep for 11 seconds 00000100:00100000:1.0:1787784768.568968:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784768.569083:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658127872 00000100:00100000:0.0:1787784768.569093:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658127872 00010000:00010000:0.0:1787784768.569143:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 14 pid: 13825 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784768.569211:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 318 00000100:00100000:0.0:1787784768.569222:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658127872:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784768.569238:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784768.569260:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 14 pid: 13825 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.569286:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 14 pid: 13825 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.569311:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784768.569365:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658127872:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 145us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784768.569374:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 318 00010000:00010000:3.0:1787784768.569650:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091f90296/0xb0ce63137acc60e8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1006 expref: 13 pid: 13825 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784768.569666:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530516580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668398976:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784768.569677:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530515a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668399104:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784768.569698:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784768.570445:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784768.570614:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658128000 00000100:00100000:2.0:1787784768.570621:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658128000 00010000:00010000:2.0:1787784768.570646:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 13 pid: 10688 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784768.570712:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 319 00000100:00100000:2.0:1787784768.570720:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658128000:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784768.570732:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784768.570748:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 13 pid: 10688 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784768.570759:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 13 pid: 10688 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784768.570784:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f035e1e6/0xb0ce63137acc60c5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0ff8 expref: 13 pid: 10688 timeout: 659 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784768.570796:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784768.570800:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784768.570805:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784768.570815:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784768.570843:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784768.570878:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658128000:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 158us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784768.570885:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 319 00010000:00010000:1.0:1787784768.570924:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.570943:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.570967:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.570976:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.570984:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.570991:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784768.571004:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.571013:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.571024:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.571031:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc6104 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784768.571085:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658127744:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 6612us (6796us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784768.571092:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00100000:2.0:1787784768.581404:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784768.581531:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658128256 00000100:00100000:1.0:1787784768.581542:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658128256 00000100:00100000:1.0:1787784768.581611:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 209 00000100:00100000:1.0:1787784768.581622:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:28101:x1874627658128256:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:1.0:1787784768.581687:0:5815:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x1d:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc60f6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b100d expref: 95 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784768.581921:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967978 00000001:00080000:1.0:1787784768.582493:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967979 00010000:00010000:1.0:1787784768.582555:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.582571:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.582581:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.582590:0:5815:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 5/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.582627:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.582659:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 98 pid: 5813 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784768.582675:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668399232:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784768.582790:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.582801:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 48 pid: 5812 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784768.582811:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668399360:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784768.582837:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527ccec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784768.585548:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784768.585589:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658128384 00000100:00100000:1.0:1787784768.585597:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658128384 00010000:00010000:1.0:1787784768.585633:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 49 pid: 5812 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784768.585672:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00100000:1.0:1787784768.585682:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658128384:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784768.585695:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784768.585702:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 49 pid: 5812 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.585725:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 49 pid: 5812 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.585755:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:1.0:1787784768.585760:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.585768:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.585775:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784768.585825:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658128384:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 144us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784768.585834:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 320 00010000:00010000:0.0:1787784768.586194:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6055 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fc0 expref: 48 pid: 5812 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784768.586211:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668399360:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784768.592710:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784768.592841:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668399232:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784768.592844:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658128512 00000100:00100000:1.0:1787784768.592851:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658128512 00010000:00010000:0.0:1787784768.592868:0:5815:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.592882:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 99 pid: 5813 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784768.592918:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 321 00000100:00100000:1.0:1787784768.592927:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:2357:x1874627658128512:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784768.592941:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784768.592948:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 99 pid: 5813 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.592960:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 99 pid: 5813 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.592982:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6086 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fd5 expref: 99 pid: 5813 timeout: 659 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.593447:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:1.0:1787784768.593452:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.593461:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.593468:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.593494:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784768.593548:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:2357:x1874627658128512:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 621us (855us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784768.593554:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 321 00010000:00010000:0.0:1787784768.593841:0:5815:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.593850:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784768.593971:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967980 00000001:00080000:0.0:1787784768.594442:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967981 00000001:00080000:0.0:1787784768.594507:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967982 00000100:00100000:0.0:1787784768.594587:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5e00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668399488:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1787784768.595336:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784768.595384:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f780 going to sleep for 11 seconds 00000100:00100000:1.0:1787784768.595618:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668399488 00000100:00100000:1.0:1787784768.595636:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668399488 00000100:00100000:1.0:1787784768.595697:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00000100:00080000:2.0:1787784768.595712:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784768.595844:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668399488:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1787784768.595934:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668399488:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 95us (601us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784768.595942:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00000100:00100000:2.0:1787784768.595985:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5e00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668399488:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:2.0:1787784768.596525:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967983 00000100:00100000:2.0:1787784768.596739:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668399616:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1787784768.596776:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784768.596804:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784768.599872:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668399616 00000100:00100000:3.0:1787784768.599887:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668399616 00000100:00100000:3.0:1787784768.599945:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:3.0:1787784768.599955:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668399616:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1787784768.600241:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668399616:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 287us (3463us total) trans 4294967358 rc 0/0 00000100:00100000:3.0:1787784768.600249:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00000100:00100000:3.0:1787784768.600305:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668399616:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:3.0:1787784768.600333:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.600344:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.600351:0:5815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.600357:0:5815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784768.600361:0:5815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.600376:0:5815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.600393:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:3.0:1787784768.600396:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:3.0:1787784768.600401:0:5815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.600407:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc610b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784768.600605:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x41:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787784768.600729:0:5815:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].14, 64 off9160 00000004:00080000:3.0:1787784768.600740:0:5815:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 559687591731ns, next at 560687589871ns, rc = 0 00000001:00080000:3.0:1787784768.600748:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967984 00010000:00010000:3.0:1787784768.600789:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989503a6d200) 00010000:00010000:3.0:1787784768.600793:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1787784768.600800:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc60f6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b100d expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784768.600891:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28101:x1874627658128256:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 19270us (19497us total) trans 4294967985 rc 0/0 00000100:00100000:3.0:1787784768.600900:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 209 00000020:00080000:3.0:1787784768.600904:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:1.0:1787784768.621630:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784768.621698:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658128768 00000100:00100000:3.0:1787784768.621717:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658128768 00000100:00100000:3.0:1787784768.621780:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1057 00000100:00100000:3.0:1787784768.621790:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28102:x1874627658128768:12345-192.168.203.43@tcp:101:cmp.0 00010000:00010000:3.0:1787784768.621825:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784768.621847:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc6112 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1014 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.621916:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.621930:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.621941:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.621954:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.621960:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.621967:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622093:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6127 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622101:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6127 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622119:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6127 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784768.622130:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:3.0:1787784768.622352:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622361:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622367:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622373:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784768.622378:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622385:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622397:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:3.0:1787784768.622400:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784768.622404:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622410:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6119 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622423:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622429:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622434:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622451:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784768.622454:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622460:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622470:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff989505542300) 00010000:00010000:3.0:1787784768.622473:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784768.622475:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622480:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6120 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622546:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc6112 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1014 expref: 104 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622561:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6127 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1014 expref: 103 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.622570:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784768.622572:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784768.622574:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784768.627775:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:28102:x1874627658128768:12345-192.168.203.43@tcp:101:cmp.0 Request processed in 5973us (6142us total) trans 4294967986 rc 0/0 00000100:00100000:3.0:1787784768.627789:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1057 00000100:00100000:0.0:1787784768.638610:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784768.638808:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658129024 00000100:00100000:3.0:1787784768.638827:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658129024 00000100:00100000:3.0:1787784768.640995:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 46 00000100:00100000:3.0:1787784768.641013:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:28102:x1874627658129024:12345-192.168.203.43@tcp:103:cmp.0 00010000:00010000:3.0:1787784768.641032:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784768.641060:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6127 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1014 expref: 103 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784768.641076:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784768.641087:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784768.641165:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:28102:x1874627658129024:12345-192.168.203.43@tcp:103:cmp.0 Request processed in 140us (2546us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784768.641174:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 46 00000100:00100000:3.0:1787784768.659699:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784768.660262:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658129152 00000100:00100000:1.0:1787784768.660272:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658129152 00000100:00100000:1.0:1787784768.660327:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1058 00000100:00100000:1.0:1787784768.660338:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:28102:x1874627658129152:12345-192.168.203.43@tcp:58:cmp.0 00010000:00010000:1.0:1787784768.660435:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.660461:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784768.660472:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787784768.660622:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].29, 136 off12136 00000100:00100000:1.0:1787784768.660711:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:28102:x1874627658129152:12345-192.168.203.43@tcp:58:cmp.0 Request processed in 365us (1009us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784768.660720:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1058 00000100:00100000:0.0:1787784768.669096:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784768.669166:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658129280 00000100:00100000:0.0:1787784768.669175:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658129280 00000100:00100000:0.0:1787784768.669227:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1059 00000100:00100000:0.0:1787784768.669237:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:28102:x1874627658129280:12345-192.168.203.43@tcp:101:cmp.0 00010000:00010000:0.0:1787784768.669258:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784768.669279:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b101b expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.669338:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.669350:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 4/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b101b expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.669370:0:5812:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.669384:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b101b expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.669397:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff98951c407400) 00010000:00010000:0.0:1787784768.669401:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784768.669404:0:5812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b101b expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784768.669412:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030e00, rc 0) 00000100:00100000:0.0:1787784768.669473:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:28102:x1874627658129280:12345-192.168.203.43@tcp:101:cmp.0 Request processed in 237us (386us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784768.669480:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1059 00000100:00100000:3.0:1787784769.010579:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784769.010619:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539250b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668399744:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784769.010745:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784769.010862:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668400128:0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.010898:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784769.010910:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668400256:0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.010982:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668400128 00000100:00100000:1.0:1787784769.010989:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784769.010995:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668400128 00000100:00100000:3.0:1787784769.011076:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:3.0:1787784769.011090:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668400128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.011200:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668400128:12345-0@lo:400:kworker.0 Request processed in 113us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.011211:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1787784769.011211:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668399872:0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.011259:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668400256 00000100:00100000:3.0:1787784769.011266:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668400256 00000100:00100000:2.0:1787784769.011295:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784769.011329:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:2.0:1787784769.011340:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668400384:0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.011341:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668400256:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784769.011385:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784769.011473:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668399744 00000100:00100000:0.0:1787784769.011479:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668399744 00000100:00100000:1.0:1787784769.011496:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668400256:0@lo:400:kworker.0 00000100:00100000:0.0:1787784769.011523:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:0.0:1787784769.011534:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668399744:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784769.011549:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668400128:0@lo:400:kworker.0 00002000:00000024:0.0:1787784769.011554:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 81592320 pending 1703936 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784769.011593:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668400384 00000100:00100000:1.0:1787784769.011600:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668400384 00000100:00100000:3.0:1787784769.011612:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668400256:12345-0@lo:400:kworker.0 Request processed in 272us (626us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.011620:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:3.0:1787784769.011649:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506337840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668400000:0@lo:400:kworker.0 00000100:00100000:1.0:1787784769.011666:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1787784769.011681:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668400384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.011691:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784769.011736:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668400512:0@lo:400:kworker.0 00000100:00100000:1.0:1787784769.011763:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668400384:12345-0@lo:400:kworker.0 Request processed in 86us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.011772:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:3.0:1787784769.011792:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784769.011793:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668400000 00000100:00100000:1.0:1787784769.011800:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668400000 00000100:00100000:1.0:1787784769.011837:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:3.0:1787784769.011844:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668400512 00000100:00100000:1.0:1787784769.011846:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668400000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.011853:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668400512 00000100:00100000:3.0:1787784769.011893:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:3.0:1787784769.011902:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668400512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784769.011917:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668400000:12345-0@lo:400:kworker.0 Request processed in 72us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.011935:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:3.0:1787784769.011954:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668400512:12345-0@lo:400:kworker.0 Request processed in 51us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.011963:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1787784769.011968:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668399872 00000100:00100000:1.0:1787784769.011975:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668399872 00000100:00100000:3.0:1787784769.012019:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506337840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668400000:0@lo:400:kworker.0 00000100:00100000:1.0:1787784769.012021:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 554 00000100:00100000:1.0:1787784769.012042:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668399872:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784769.012046:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668400512:0@lo:400:kworker.0 00000100:00100000:1.0:1787784769.012105:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668399872:12345-0@lo:400:kworker.0 Request processed in 66us (811us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.012114:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 554 00000100:00080000:3.0:1787784769.012144:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784769.012187:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784769.017236:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668400384:0@lo:400:kworker.0 00000100:00100000:2.0:1787784769.017286:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668399872:0@lo:400:kworker.0 00000100:00100000:0.0:1787784769.020562:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668399744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 9017us (9804us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784769.020582:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:1.0:1787784769.020794:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539250b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668399744:0@lo:13:osp-pre-1-0.0 00000040:00080000:2.0:1787784769.076240:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784769.076259:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784769.076302:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 30 count 5 00000040:00080000:2.0:1787784769.076310:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784769.076442:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251a40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668400640:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784769.076519:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784769.082430:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251a40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668400640:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784769.084373:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.084646:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658129408 00000100:00100000:1.0:1787784769.084657:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658129408 00000100:00100000:1.0:1787784769.084773:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1060 00000100:00100000:1.0:1787784769.084786:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28103:x1874627658129408:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784769.084836:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784769.084984:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28103:x1874627658129408:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 200us (623us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.084993:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1060 00000100:00080000:3.0:1787784769.085070:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784769.096962:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.097554:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658129536 00000100:00100000:1.0:1787784769.097570:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658129536 00000100:00100000:1.0:1787784769.097946:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1061 00000100:00100000:1.0:1787784769.097960:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28103:x1874627658129536:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1787784769.098060:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098082:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098120:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098401:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098425:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098438:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098481:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098491:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098498:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784769.098539:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00000004:00200000:1.0:1787784769.098543:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784769.098557:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098569:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098575:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098671:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098677:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784769.098681:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098690:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098712:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784769.098716:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784769.098721:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098728:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6143 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098745:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098751:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098756:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098761:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784769.098765:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098781:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098790:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3e15321 (ffff98951c407700) 00010000:00010000:1.0:1787784769.098794:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784769.098798:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098805:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc614a lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098827:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098834:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098840:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098847:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784769.098850:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098857:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098868:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff98951c407400) 00010000:00010000:1.0:1787784769.098871:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784769.098875:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.098892:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6151 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.098997:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28103:x1874627658129536:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 1037us (2047us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.099007:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1061 00000100:00100000:2.0:1787784769.101393:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.101532:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658129792 00000100:00100000:3.0:1787784769.101545:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658129792 00000100:00100000:3.0:1787784769.101621:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1062 00000100:00100000:3.0:1787784769.101633:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28103:x1874627658129792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784769.101662:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784769.101721:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc6158 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1022 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103004:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103028:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103044:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784769.103057:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:3.0:1787784769.103206:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103215:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103221:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103227:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784769.103232:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103239:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103253:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff98951c407400) 00010000:00010000:3.0:1787784769.103257:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784769.103263:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103277:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc615f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103336:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc6158 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1022 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103349:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784769.103351:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784769.103355:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc6158 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1022 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.103365:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6e00, rc 0) 00000100:00100000:3.0:1787784769.103463:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28103:x1874627658129792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1831us (2083us total) trans 4294967987 rc 301/301 00000100:00100000:3.0:1787784769.103473:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1062 00000100:00100000:0.0:1787784769.108707:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.108854:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658129920 00000100:00100000:3.0:1787784769.108864:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658129920 00000100:00100000:3.0:1787784769.108924:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1063 00000100:00100000:3.0:1787784769.108933:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28103:x1874627658129920:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784769.108969:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784769.109070:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28103:x1874627658129920:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 138us (382us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.109078:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1063 00000100:00100000:1.0:1787784769.119539:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.119671:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658130176 00000100:00100000:3.0:1787784769.119682:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658130176 00000100:00100000:3.0:1787784769.119757:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1064 00000100:00100000:3.0:1787784769.119781:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28103:x1874627658130176:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784769.119803:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784769.119824:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc616d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1029 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.120597:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6174 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.120613:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6174 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.120625:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6174 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784769.120636:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 0 00010000:00010000:3.0:1787784769.120682:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6e00/0xb0ce63137acc616d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1029 expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.120716:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6174 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1029 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.120725:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784769.120728:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784769.120731:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784769.120831:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28103:x1874627658130176:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1051us (1304us total) trans 4294967988 rc 0/0 00000100:00100000:3.0:1787784769.120839:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1064 00000100:00100000:2.0:1787784769.124597:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.124785:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658130432 00000100:00100000:3.0:1787784769.124794:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658130432 00000100:00100000:3.0:1787784769.124899:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1065 00000100:00100000:3.0:1787784769.124909:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28103:x1874627658130432:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784769.124962:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784769.124987:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6174 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1029 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.125006:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989505542800) 00010000:00010000:3.0:1787784769.125016:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784769.125037:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.125050:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.125058:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.125089:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.125102:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 102 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784769.125115:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ecb40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668400768:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784769.125228:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1180 going to sleep for 11 seconds 00000100:00100000:0.0:1787784769.128597:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.129010:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ecb40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668400768:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784769.129034:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784769.129116:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658130560 00000100:00100000:2.0:1787784769.129123:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658130560 00010000:00010000:2.0:1787784769.129237:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 103 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784769.129271:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 322 00000100:00100000:2.0:1787784769.129281:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:2356:x1874627658130560:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784769.129305:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784769.129311:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 103 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129335:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 103 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129357:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc609b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0fe3 expref: 103 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129372:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784769.129377:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129387:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129395:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129417:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784769.129464:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:2356:x1874627658130560:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 184us (874us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784769.129472:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 322 00010000:00010000:2.0:1787784769.129494:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129501:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129564:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129584:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129592:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129668:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129688:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129818:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129827:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784769.129831:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129839:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129848:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784769.129850:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784769.129853:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129859:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6182 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129868:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129879:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129885:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129889:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784769.129892:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129898:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129904:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0xaaa14283 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.129906:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784769.129908:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.129913:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6189 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0xaaa14283 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784769.129963:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:28103:x1874627658130432:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 5055us (5373us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787784769.129970:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1065 00000100:00080000:3.0:1787784769.132360:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784769.138454:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784769.138964:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658130688 00000100:00100000:2.0:1787784769.138975:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658130688 00000100:00100000:2.0:1787784769.139037:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1066 00000100:00100000:2.0:1787784769.139049:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:28103:x1874627658130688:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784769.139069:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784769.139105:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6190 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1030 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.139141:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.139190:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.139201:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.139304:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6190 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 52 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.139320:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.139328:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.139331:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784769.139335:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784769.139387:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28103:x1874627658130688:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 340us (942us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784769.139395:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1066 00000100:00100000:2.0:1787784769.143068:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.143186:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658130944 00000100:00100000:1.0:1787784769.143196:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658130944 00000100:00100000:1.0:1787784769.143245:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1067 00000100:00100000:1.0:1787784769.143255:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28103:x1874627658130944:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784769.143340:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.143356:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.143366:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784769.143549:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f210271d reply state 00000000318b1c51 transno 4294967989 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.143559:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784769.143875:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:28103:x1874627658130944:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 620us (814us total) trans 4294967989 rc 0/0 00000100:00100000:1.0:1787784769.143883:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1067 00000100:00100000:0.0:1787784769.147866:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.148006:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658131072 00000100:00100000:1.0:1787784769.148029:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658131072 00000100:00100000:1.0:1787784769.148077:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1068 00000100:00100000:1.0:1787784769.148088:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:28103:x1874627658131072:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784769.148111:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784769.148133:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1037 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.148209:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 54 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.148224:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 54 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.148235:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784769.148238:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784769.148242:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784769.148309:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:28103:x1874627658131072:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 223us (451us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.148317:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1068 00010000:00010000:2.0:1787784769.148520:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.148530:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.148549:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.148556:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784769.148560:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.148568:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.148581:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.148583:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787784769.148586:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.148592:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc619e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.151562:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.151677:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658131200 00000100:00100000:3.0:1787784769.151687:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658131200 00000100:00100000:3.0:1787784769.151755:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:3.0:1787784769.151766:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d5450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:28103:x1874627658131200:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1787784769.151896:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d5450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:28103:x1874627658131200:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 130us (342us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.151904:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 210 00000100:00100000:0.0:1787784769.164595:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.165672:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658131456 00000100:00100000:3.0:1787784769.165688:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658131456 00000100:00100000:3.0:1787784769.165794:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 211 00000100:00100000:3.0:1787784769.165806:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:28103:x1874627658131456:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784769.166088:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:28103:x1874627658131456:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 280us (1498us total) trans 4294967990 rc 0/0 00000100:00100000:3.0:1787784769.166097:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 211 00000100:00100000:0.0:1787784769.176416:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.176505:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658131712 00000100:00100000:1.0:1787784769.176514:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658131712 00000100:00100000:1.0:1787784769.176572:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1069 00000100:00100000:1.0:1787784769.176581:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:28103:x1874627658131712:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784769.176710:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:28103:x1874627658131712:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 118us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.176718:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1069 00000100:00100000:1.0:1787784769.179950:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784769.180049:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658131968 00000100:00100000:2.0:1787784769.180059:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658131968 00000100:00100000:2.0:1787784769.180116:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1070 00000100:00100000:2.0:1787784769.180124:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:28103:x1874627658131968:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784769.180144:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784769.180197:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc61ac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b103e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.180304:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.180339:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.180348:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.180380:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.180394:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 55 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784769.180407:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668400896:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784769.180474:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784769.186779:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.186859:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658132096 00000100:00100000:1.0:1787784769.186868:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658132096 00010000:00010000:1.0:1787784769.186906:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 56 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.186944:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 323 00000100:00100000:1.0:1787784769.186953:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+56:2357:x1874627658132096:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784769.186970:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784769.186976:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 56 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.186989:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 56 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.187012:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784769.187018:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.187027:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.187035:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.187052:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784769.187104:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+56:2357:x1874627658132096:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 151us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.187112:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 323 00010000:00010000:2.0:1787784769.187391:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc61a5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1037 expref: 55 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784769.187408:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668400896:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784769.187425:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187434:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187440:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784769.187459:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 0 00000004:00080000:2.0:1787784769.187643:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x15:0x0], index 0: 0 00010000:00010000:2.0:1787784769.187808:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187817:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187824:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187829:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784769.187834:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187841:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187853:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.187856:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784769.187859:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187864:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61b3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187909:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc61ac lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b103e expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187918:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff989503a6de00) 00010000:00010000:2.0:1787784769.187920:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784769.187923:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc61ac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b103e expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.187942:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784769.187986:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:28103:x1874627658131968:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 7863us (8045us total) trans 4294967991 rc 301/301 00000100:00100000:2.0:1787784769.187994:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1070 00000100:00100000:0.0:1787784769.191888:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.191964:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658132352 00000100:00100000:3.0:1787784769.191975:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658132352 00000100:00100000:3.0:1787784769.192032:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 212 00000100:00100000:3.0:1787784769.192042:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:28103:x1874627658132352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784769.192425:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+55:28103:x1874627658132352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 383us (546us total) trans 4294967992 rc 0/0 00000100:00100000:3.0:1787784769.192434:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 212 00000100:00100000:0.0:1787784769.195005:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.195082:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658132480 00000100:00100000:1.0:1787784769.195091:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658132480 00000100:00100000:1.0:1787784769.195138:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1071 00000100:00100000:1.0:1787784769.195175:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+55:28103:x1874627658132480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784769.195194:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784769.195213:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61c1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1045 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.195273:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc61c8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.195295:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc61c8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.195306:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc61c8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784769.195335:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 0 00000004:00200000:1.0:1787784769.195339:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784769.195353:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc61c8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.195377:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61c1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1045 expref: 57 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.195394:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc61c8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1045 expref: 56 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.195402:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784769.195405:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1787784769.195415:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:1.0:1787784769.195454:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+56:28103:x1874627658132480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 309us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.195461:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1071 00000100:00100000:0.0:1787784769.198341:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.198480:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658132608 00000100:00100000:1.0:1787784769.198490:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658132608 00000100:00100000:1.0:1787784769.198558:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:1.0:1787784769.198567:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658132608:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784769.198584:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784769.198605:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b104c expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784769.198631:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784769.198650:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=560s v=(0 0) 00010000:00010000:1.0:1787784769.198654:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784769.198664:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787784769.198847:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787784769.198893:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.198906:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d800, rc 0) 00000100:00100000:1.0:1787784769.199926:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658132608:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1355us (1589us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.199936:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:0.0:1787784769.202369:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.202556:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658132864 00000100:00100000:1.0:1787784769.202579:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658132864 00000100:00100000:1.0:1787784769.202634:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1072 00000100:00100000:1.0:1787784769.202644:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:28103:x1874627658132864:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784769.202831:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:28103:x1874627658132864:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 189us (470us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.202839:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1072 00000100:00100000:0.0:1787784769.205927:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.206013:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658132992 00000100:00100000:3.0:1787784769.206023:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658132992 00000100:00100000:3.0:1787784769.206094:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:3.0:1787784769.206104:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:28103:x1874627658132992:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784769.206120:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784769.206138:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1053 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.209999:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.210012:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784769.210017:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784769.210023:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.210062:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.210078:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 10 pid: 10688 timeout: 660 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784769.210094:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668401024:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784769.210196:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784769.216212:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.216301:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658133120 00000100:00100000:1.0:1787784769.216310:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658133120 00010000:00010000:1.0:1787784769.216345:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 11 pid: 10688 timeout: 660 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.216373:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:1.0:1787784769.216381:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658133120:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784769.216395:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784769.216414:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 11 pid: 10688 timeout: 660 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.216428:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 11 pid: 10688 timeout: 660 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.216452:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784769.216456:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784769.216464:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.216475:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.216519:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.216532:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668401152:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784769.216592:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce180 going to sleep for 11 seconds 00010000:00010000:3.0:1787784769.216660:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e3ac749/0xb0ce63137acc61cf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b104c expref: 11 pid: 10688 timeout: 660 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784769.216674:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ec3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668401024:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784769.216717:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.216728:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784769.216753:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:28103:x1874627658132992:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 10652us (10834us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.216760:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:1.0:1787784769.222405:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668401152:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784769.222430:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784769.222471:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658133120:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 6090us (6266us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.222481:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:0.0:1787784769.224089:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.224192:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658133248 00000100:00100000:3.0:1787784769.224200:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658133248 00002000:00100000:3.0:1787784769.224242:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x15:0x0] (0->1515) req@ffff989404dbd2c0 x1874627658133248/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:650/0 lens 2008/0 e 0 to 0 dl 1787784780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784769.224253:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbd2c0 with x1874627658133248 ext(0->1515) 00002000:00010000:3.0:1787784769.224258:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989404dbd2c0 00000100:00100000:3.0:1787784769.224292:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 97 00000100:00100000:3.0:1787784769.224300:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658133248:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1787784769.224776:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658133248:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 474us (691us total) trans 4294967351 rc 0/0 00000100:00100000:3.0:1787784769.224784:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 97 00002000:00100000:3.0:1787784769.224791:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x15:0x0] (0->1515) req@ffff989404dbd2c0 x1874627658133248/t4294967351(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:650/0 lens 2008/448 e 0 to 0 dl 1787784780 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784769.224799:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbd2c0 with x1874627658133248 ext(0->1515) 00002000:00010000:3.0:1787784769.224802:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989404dbd2c0 00000100:00100000:0.0:1787784769.233114:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.233518:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658133376 00000100:00100000:1.0:1787784769.233527:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658133376 00000100:00100000:1.0:1787784769.233643:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1073 00000100:00100000:1.0:1787784769.233653:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+56:28103:x1874627658133376:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:1.0:1787784769.276872:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967983 is committed 00000001:00080000:1.0:1787784769.276894:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967976 is committed 00000001:00080000:1.0:1787784769.276909:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967989 is committed 00000001:00080000:1.0:1787784769.276926:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967991 is committed 00000001:00080000:1.0:1787784769.276933:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967984 is committed 00000020:00080000:1.0:1787784769.276939:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989527cf3e40 refcount 2 committed 0 result 0 batchid 19 00000020:00080000:1.0:1787784769.276944:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953f510c00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989527cf3540 00000020:00080000:1.0:1787784769.276948:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953f510d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff98940297cc00 00000001:00080000:1.0:1787784769.276967:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967992 is committed 00000001:00080000:1.0:1787784769.276991:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967986 is committed 00000004:00080000:1.0:1787784769.276996:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 15 committed 00000020:00080000:1.0:1787784769.277018:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 19 committed batchid 18 00000020:00080000:1.0:1787784769.277219:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 18 00000100:00100000:1.0:1787784769.277338:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668401280:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1787784769.277402:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784769.277459:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668401280 00000100:00100000:1.0:1787784769.277466:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668401280 00000100:00100000:1.0:1787784769.277508:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1787784769.277517:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668401280:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1787784769.277543:0:6670:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:23 count 1 00002000:00010000:1.0:1787784769.277575:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x17:0x0] initial LVB size: 1516, mtime: 0x6a8f6e3f, atime: 0x0, ctime: 0x6a8f6e3f, blocks: 0x1 00010000:00010000:1.0:1787784769.277584:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 3/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.277600:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=560s v=(0 0) 00010000:00010000:1.0:1787784769.277605:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784769.277612:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.277629:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.277639:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.277647:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.277672:0:6670:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787784769.278107:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3278:x1874627668401280:12345-0@lo:6:osp-syn-1-0.0 Request processed in 590us (704us total) trans 4294967359 rc 0/0 00000100:00100000:1.0:1787784769.278115:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00010000:00010000:1.0:1787784769.278175:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784769.278184:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.278194:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784769.278200:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.278211:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.278233:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 1/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.278243:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001bcfaae0/0xb0ce63137acc61dd lrc: 0/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.278263:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784769.278265:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784769.278267:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787784769.278299:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff98953f7b1680/2, rc 0, transno 4294967359 00000100:00100000:1.0:1787784769.278305:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668401280:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787784769.278541:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28103:x1874627658133376:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 44887us (45438us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784769.278550:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1073 00000100:00100000:0.0:1787784769.283191:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.283535:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658133504 00000100:00100000:3.0:1787784769.283546:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658133504 00000100:00100000:3.0:1787784769.283618:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 288 00000100:00100000:3.0:1787784769.283628:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658133504:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1787784769.314264:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658133504:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 30622us (31068us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.314278:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 288 00000001:00080000:0.0:1787784769.314823:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967351 is committed 00000100:00100000:0.0:1787784769.318509:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.318609:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658133760 00000100:00100000:1.0:1787784769.318619:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658133760 00000100:00100000:1.0:1787784769.318826:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1074 00000100:00100000:1.0:1787784769.318837:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28103:x1874627658133760:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784769.318888:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784769.318915:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc61c8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1045 expref: 44 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.318935:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784769.318941:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784769.318970:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.318986:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.318995:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784769.319246:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007c0e4b9f reply state 00000000ac1aad8a transno 4294967993 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.319256:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784769.319363:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28103:x1874627658133760:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 527us (862us total) trans 4294967993 rc 0/0 00000100:00100000:1.0:1787784769.319371:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1074 00010000:00010000:0.0:1787784769.321910:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.321926:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.321933:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.321939:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784769.321944:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.321952:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.321970:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:0.0:1787784769.321975:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784769.321980:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.321987:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc61e4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784769.322975:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784769.324356:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658133888 00000100:00100000:2.0:1787784769.324369:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658133888 00000100:00100000:2.0:1787784769.324461:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1075 00000100:00100000:2.0:1787784769.324473:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28103:x1874627658133888:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784769.324497:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784769.324517:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b105a expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.324574:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.324586:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.324605:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.324608:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784769.324612:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784769.324680:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28103:x1874627658133888:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 199us (1699us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784769.324689:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1075 00000100:00100000:0.0:1787784769.328611:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.328757:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658134016 00000100:00100000:3.0:1787784769.328767:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658134016 00000100:00100000:3.0:1787784769.328855:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 289 00000100:00100000:3.0:1787784769.328865:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658134016:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1787784769.329438:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 0 -> 1516 00002000:00010000:3.0:1787784769.329444:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb mtime from disk: 0 -> 1787784767 00002000:00010000:3.0:1787784769.329447:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb atime from disk: 0 -> 1787784766 00002000:00010000:3.0:1787784769.329449:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb ctime from disk: 0 -> 1787784767 00002000:00010000:3.0:1787784769.329452:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1787784769.329818:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658134016:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 952us (1214us total) trans 4294967352 rc 0/0 00000100:00100000:0.0:1787784769.329827:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 289 00000100:00100000:2.0:1787784769.333680:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784769.333812:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658134272 00000100:00100000:0.0:1787784769.333822:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658134272 00000100:00100000:0.0:1787784769.333879:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1076 00000100:00100000:0.0:1787784769.333888:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28103:x1874627658134272:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1787784769.333951:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.333969:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.333978:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.334011:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.334024:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 45 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784769.334038:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbe940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668401408:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784769.334129:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784769.337691:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784769.337914:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658134400 00000100:00100000:2.0:1787784769.337924:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658134400 00010000:00010000:2.0:1787784769.337961:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 46 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784769.338003:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:2.0:1787784769.338013:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658134400:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784769.338028:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784769.338034:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 46 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.338046:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 46 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.338068:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.338073:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.338082:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.338089:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.338107:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784769.338191:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658134400:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 176us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784769.338201:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 325 00010000:00010000:1.0:1787784769.338843:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6197 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1030 expref: 45 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.338876:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbe940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668401408:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784769.338895:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338904:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338910:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338937:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338944:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338950:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338955:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784769.338959:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338966:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338976:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784769.338979:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787784769.338982:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.338994:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc61f2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.339057:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28103:x1874627658134272:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 5169us (5385us total) trans 0 rc -22/-22 00000100:00100000:1.0:1787784769.339066:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1076 00000100:00100000:0.0:1787784769.342568:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.342626:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658134528 00000100:00100000:1.0:1787784769.342635:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658134528 00000100:00100000:1.0:1787784769.342681:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:1.0:1787784769.342690:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658134528:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784769.342743:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.342763:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.342772:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784769.342776:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784769.342781:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.342810:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.342824:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 10 pid: 13825 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.342857:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668401536:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784769.342938:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce540 going to sleep for 11 seconds 00000100:00100000:3.0:1787784769.346775:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784769.346855:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658134656 00000100:00100000:0.0:1787784769.346878:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658134656 00010000:00010000:0.0:1787784769.346915:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 11 pid: 13825 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784769.346963:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 326 00000100:00100000:0.0:1787784769.346977:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658134656:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784769.346997:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784769.347025:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 11 pid: 13825 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.347043:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 11 pid: 13825 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.347073:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784769.347079:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784769.347086:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.347097:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.347122:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784769.347230:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658134656:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 257us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784769.347238:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 326 00010000:00010000:1.0:1787784769.347390:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc61d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1053 expref: 10 pid: 13825 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.347405:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668401536:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784769.347424:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347434:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347442:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347464:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347473:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347481:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347500:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784769.347504:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347513:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347525:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.347532:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e5c7518/0xb0ce63137acc61f9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.347572:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658134528:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 4884us (5013us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.347580:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:2.0:1787784769.351139:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784769.351288:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658134912 00000100:00100000:0.0:1787784769.351297:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658134912 00000100:00100000:0.0:1787784769.351358:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1077 00000100:00100000:0.0:1787784769.351366:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28103:x1874627658134912:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784769.351434:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.351451:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.351462:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.351639:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.351650:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.351658:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.351687:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.351717:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 45 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784769.351731:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668401664:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784769.351806:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784769.355565:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784769.355640:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658135040 00000100:00100000:1.0:1787784769.355648:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658135040 00010000:00010000:1.0:1787784769.355688:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 46 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.355734:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 327 00000100:00100000:1.0:1787784769.355742:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658135040:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784769.355756:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784769.355773:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 46 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.355786:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 46 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.355814:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:1.0:1787784769.355819:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.355828:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.355836:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.355853:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784769.355900:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658135040:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 159us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784769.355907:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 327 00010000:00010000:0.0:1787784769.357353:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc61eb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b105a expref: 45 pid: 5811 timeout: 660 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784769.357389:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668401664:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784769.357409:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.357419:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784769.357425:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784769.357579:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967994 00000001:00080000:0.0:1787784769.357823:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967995 00000001:00080000:0.0:1787784769.357866:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967996 00000100:00100000:0.0:1787784769.357936:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668401792:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787784769.357986:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784769.358070:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f300 going to sleep for 11 seconds 00000100:00100000:1.0:1787784769.358115:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668401792 00000100:00100000:1.0:1787784769.358121:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668401792 00000100:00100000:0.0:1787784769.358416:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:0.0:1787784769.358427:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668401792:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787784769.358505:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668401792:12345-0@lo:1000:mdt00_001.0 Request processed in 80us (518us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784769.358512:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00000100:00100000:1.0:1787784769.358573:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668401792:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1787784769.358925:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967997 00000100:00100000:1.0:1787784769.358978:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668401920:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787784769.359025:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784769.359479:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668401920 00000100:00100000:3.0:1787784769.359485:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668401920 00000100:00100000:3.0:1787784769.359517:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:3.0:1787784769.359524:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668401920:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787784769.359712:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5812:x1874627668401920:12345-0@lo:1000:mdt00_001.0 Request processed in 187us (686us total) trans 4294967353 rc 0/0 00000100:00100000:3.0:1787784769.359719:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1787784769.359968:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784769.359994:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668401920:0@lo:1000:mdt00_001.0 00000001:00080000:2.0:1787784769.360077:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967998 00010000:00010000:2.0:1787784769.360108:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360118:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360125:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360131:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784769.360136:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360262:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360279:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.360351:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784769.360358:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360366:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6207 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360398:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360419:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360425:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360432:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784769.360435:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360441:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360451:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff98951c407400) 00010000:00010000:2.0:1787784769.360454:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784769.360460:0:5812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b101b expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360472:0:5812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b101b expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360482:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b101b expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.360510:0:5812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500030e00/0xb0ce63137acc613c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b101b expref: 90 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784769.360523:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668402048:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784769.360642:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950677b600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784769.364247:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784769.364271:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658135168 00000100:00100000:2.0:1787784769.364278:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658135168 00000100:00100000:2.0:1787784769.364329:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 213 00000100:00100000:2.0:1787784769.364338:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28102:x1874627658135168:12345-192.168.203.43@tcp:49:cmp.0 00000100:00100000:2.0:1787784769.364443:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28102:x1874627658135168:12345-192.168.203.43@tcp:49:cmp.0 Request processed in 106us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784769.364450:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 213 00000100:00100000:1.0:1787784769.364676:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668402048:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784769.364713:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364740:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6135 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364770:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364783:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364793:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364803:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364808:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364814:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364821:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784769.364825:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364832:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364842:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff98951c407400) 00010000:00010000:1.0:1787784769.364845:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784769.364848:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364859:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc620e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364869:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364875:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364881:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364886:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784769.364889:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364894:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364901:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff98951c407400) 00010000:00010000:1.0:1787784769.364903:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784769.364906:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784769.364911:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6200 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784769.365080:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28103:x1874627658134912:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 13713us (13949us total) trans 4294967999 rc 0/0 00000100:00100000:1.0:1787784769.365088:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1077 00000020:00080000:1.0:1787784769.365096:0:5812:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:2.0:1787784769.365601:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784769.365606:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784769.366228:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:2.0:1787784769.366243:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784769.373123:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.373268:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658135296 00000100:00100000:3.0:1787784769.373279:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658135296 00000100:00100000:3.0:1787784769.373357:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:3.0:1787784769.373367:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:28102:x1874627658135296:12345-192.168.203.43@tcp:101:cmp.0 00010000:00010000:3.0:1787784769.373406:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784769.373431:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d13f86d1/0xb0ce63137acc6215 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1061 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784769.373460:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 1516, mtime: 0x6a8f6e3f, atime: 0x6a8f6e3e, ctime: 0x6a8f6e3f, blocks: 0xb 00010000:00010000:3.0:1787784769.373484:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=560s v=(0 0) 00010000:00010000:3.0:1787784769.373490:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784769.373502:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d13f86d1/0xb0ce63137acc6215 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1061 expref: 15 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.373522:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d13f86d1/0xb0ce63137acc6215 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1061 expref: 15 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784769.373537:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6e00, rc 0) 00000100:00100000:3.0:1787784769.373625:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:28102:x1874627658135296:12345-192.168.203.43@tcp:101:cmp.0 Request processed in 258us (508us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784769.373635:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00080000:2.0:1787784769.374718:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784769.377264:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784769.377329:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784769.377402:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658135680 00000100:00100000:0.0:1787784769.377411:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658135680 00002000:00100000:0.0:1787784769.377454:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x15:0x0] (0->4095) req@ffff9894030ec3c0 x1874627658135680/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:650/0 lens 488/0 e 0 to 0 dl 1787784780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784769.377466:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894030ec3c0 with x1874627658135680 ext(0->4095) 00002000:00010000:0.0:1787784769.377479:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894030ec3c0 00000100:00100000:0.0:1787784769.377504:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 98 00000100:00100000:0.0:1787784769.377512:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658135680:12345-192.168.203.43@tcp:3:cmp.0 00000100:00100000:0.0:1787784769.377762:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658135680:12345-192.168.203.43@tcp:3:cmp.0 Request processed in 246us (429us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784769.377772:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 98 00002000:00100000:0.0:1787784769.377781:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x15:0x0] (0->4095) req@ffff9894030ec3c0 x1874627658135680/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:650/0 lens 488/4536 e 0 to 0 dl 1787784780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784769.377790:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894030ec3c0 with x1874627658135680 ext(0->4095) 00002000:00010000:0.0:1787784769.377795:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894030ec3c0 00000100:00100000:2.0:1787784769.377892:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658135552 00000100:00100000:2.0:1787784769.377899:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658135552 00000100:00100000:2.0:1787784769.377959:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 214 00000100:00100000:2.0:1787784769.377971:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28103:x1874627658135552:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1787784769.378508:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1787784769.378600:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968000 00010000:00010000:2.0:1787784769.378643:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378660:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378670:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378684:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378690:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378695:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378723:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787784769.378784:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28103:x1874627658135552:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 817us (1528us total) trans 4294968001 rc 0/0 00000100:00100000:2.0:1787784769.378792:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 214 00000020:00080000:2.0:1787784769.378796:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1787784769.378817:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784769.378822:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378829:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784769.378833:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378843:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378865:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff989530cb1200) 00010000:00010000:2.0:1787784769.378870:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787784769.378875:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378896:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc621c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784769.378918:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784769.378920:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784769.378922:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784769.393658:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784769.393775:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658135936 00000100:00100000:2.0:1787784769.393784:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658135936 00000100:00100000:2.0:1787784769.393829:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 215 00000100:00100000:2.0:1787784769.393839:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28102:x1874627658135936:12345-192.168.203.43@tcp:35:cmp.0 00000100:00100000:2.0:1787784769.394052:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28102:x1874627658135936:12345-192.168.203.43@tcp:35:cmp.0 Request processed in 214us (400us total) trans 4294968002 rc 0/0 00000100:00100000:2.0:1787784769.394060:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 215 00010000:00010000:1.0:1787784769.457369:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784770.484232:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787784770.599321:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967359 is committed 00000001:00080000:2.0:1787784770.599339:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967357 is committed 00010000:00010000:2.0:1787784771.505365:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784772.082394:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668402176:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784772.082495:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784772.082639:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668402176 00000100:00100000:2.0:1787784772.082648:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668402176 00000100:00100000:2.0:1787784772.082732:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1787784772.082743:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668402176:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784772.082807:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784772.083027:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668402176:12345-0@lo:13:osp-pre-0-0.0 Request processed in 285us (531us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784772.083037:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00080000:1.0:1787784772.083299:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784772.083350:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668402176:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784772.359446:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784772.359776:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784772.359943:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784772.359978:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784772.360001:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784772.360023:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784772.360065:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784772.360101:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136320 00000100:00100000:3.0:1787784772.360113:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136320 00000100:00100000:0.0:1787784772.360182:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136064 00000100:00100000:0.0:1787784772.360188:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136064 00000100:00100000:3.0:1787784772.360203:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136832 00000100:00100000:3.0:1787784772.360206:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136832 00000100:00100000:3.0:1787784772.360222:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136448 00000100:00100000:3.0:1787784772.360224:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136448 00000100:00100000:3.0:1787784772.360235:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136704 00000100:00100000:3.0:1787784772.360237:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136704 00000100:00100000:3.0:1787784772.360254:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:3.0:1787784772.360265:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2354:x1874627658136320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784772.360364:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2354:x1874627658136320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 100us (588us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784772.360372:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:3.0:1787784772.360390:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:3.0:1787784772.360397:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658136832:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784772.360412:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 555 00000100:00100000:0.0:1787784772.360420:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658136064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784772.360423:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658136832:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (445us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784772.360429:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:3.0:1787784772.360440:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00100000:3.0:1787784772.360446:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658136448:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784772.360447:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658136064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (1017us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784772.360453:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 555 00000100:00100000:3.0:1787784772.360465:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658136448:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784772.360470:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00100000:3.0:1787784772.360479:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 332 00000100:00100000:3.0:1787784772.360499:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658136704:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784772.360699:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658136704:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 201us (634us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784772.360706:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 332 00000100:00100000:3.0:1787784772.360733:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136576 00000100:00100000:3.0:1787784772.360736:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136576 00000100:00100000:3.0:1787784772.362718:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136192 00000100:00100000:3.0:1787784772.362738:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136192 00000100:00100000:0.0:1787784772.363575:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 378 00000100:00100000:0.0:1787784772.363596:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2354:x1874627658136576:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784772.364715:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2354:x1874627658136576:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1116us (4761us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784772.364730:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 378 00000100:00100000:0.0:1787784772.364774:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 379 00000100:00100000:0.0:1787784772.364787:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:2354:x1874627658136192:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784772.365031:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:2354:x1874627658136192:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 245us (5025us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784772.365038:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 379 00000100:00080000:3.0:1787784772.365299:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784772.365307:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784772.365311:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:0.0:1787784772.466995:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784772.467009:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784772.467486:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 30 count 3 00000040:00080000:0.0:1787784772.467498:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784772.538199:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784773.559961:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784774.130531:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668402432:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.130634:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784774.130711:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668402944:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.130738:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784774.130772:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1787784774.130777:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668402304:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.130807:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784774.130826:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668402816:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.130845:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784774.130874:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668402560:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.130900:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784774.130919:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784774.130925:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668402688:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.130946:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784774.130985:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668402432 00000100:00100000:1.0:1787784774.130995:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668402432 00000100:00100000:1.0:1787784774.131053:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668402816 00000100:00100000:1.0:1787784774.131057:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668402816 00000100:00100000:1.0:1787784774.131082:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668402560 00000100:00100000:1.0:1787784774.131096:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668402560 00000100:00100000:1.0:1787784774.131120:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1787784774.131129:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668402432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131222:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668402432:12345-0@lo:400:kworker.0 Request processed in 93us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784774.131229:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1787784774.131248:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:1.0:1787784774.131254:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668402816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131279:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668402816:12345-0@lo:400:kworker.0 Request processed in 26us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784774.131285:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:1.0:1787784774.131294:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:1.0:1787784774.131299:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668402560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131326:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668402560:12345-0@lo:400:kworker.0 Request processed in 26us (425us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784774.131332:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:1.0:1787784774.131350:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668402944 00000100:00100000:1.0:1787784774.131354:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668402944 00000100:00100000:1.0:1787784774.131380:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668402688 00000100:00100000:1.0:1787784774.131383:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668402688 00000100:00100000:1.0:1787784774.131404:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:1.0:1787784774.131410:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668402944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131442:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668402944:12345-0@lo:400:kworker.0 Request processed in 33us (704us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784774.131448:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:1.0:1787784774.131459:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1787784774.131465:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668402688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131499:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668402688:12345-0@lo:400:kworker.0 Request processed in 35us (552us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784774.131505:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1787784774.131524:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668402304 00000100:00100000:1.0:1787784774.131526:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668402304 00000100:00100000:1.0:1787784774.131549:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 556 00000100:00100000:1.0:1787784774.131555:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668402304:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131587:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668402304:12345-0@lo:400:kworker.0 Request processed in 32us (779us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784774.131593:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 556 00000100:00080000:1.0:1787784774.131636:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784774.131641:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784774.131644:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784774.131686:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668402304:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131702:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540739e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668402816:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131712:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668402432:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131721:0:3277:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967359 gen 2 00000004:00080000:1.0:1787784774.131727:0:3277:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967358 committed transno 4294967359 00000004:00080000:1.0:1787784774.131751:0:3277:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff98953f7b1680, transno 4294967359 00000100:00100000:1.0:1787784774.131764:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668402944:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131798:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668402688:0@lo:400:kworker.0 00000100:00100000:1.0:1787784774.131806:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668402560:0@lo:400:kworker.0 00000100:00100000:0.0:1787784774.138056:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539250b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668403072:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784774.138191:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784774.138301:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668403072 00000100:00100000:2.0:1787784774.138311:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668403072 00000100:00100000:2.0:1787784774.138380:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:2.0:1787784774.138411:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668403072:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784774.138463:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 79888384 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784774.138524:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668403072:12345-0@lo:13:osp-pre-1-0.0 Request processed in 115us (332us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784774.138532:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:2.0:1787784774.138597:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539250b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668403072:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1787784774.577313:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784774.604018:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784774.604224:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658136960 00000100:00100000:1.0:1787784774.604234:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658136960 00000100:00100000:1.0:1787784774.604322:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1078 00000100:00100000:1.0:1787784774.604333:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28083:x1874627658136960:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1787784774.604437:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28083:x1874627658136960:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 106us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784774.604446:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1078 00000100:00080000:3.0:1787784774.607282:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1787784774.739202:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967353 is committed 00000001:00080000:0.0:1787784774.739224:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967352 is committed 00000001:00080000:1.0:1787784774.772947:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967997 is committed 00000001:00080000:1.0:1787784774.773097:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967998 is committed 00000020:00080000:1.0:1787784774.773106:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989527cce180 refcount 2 committed 0 result 0 batchid 20 00000020:00080000:1.0:1787784774.773113:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895053f0e00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989527cce540 00000020:00080000:1.0:1787784774.773116:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895053f0700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989538a24800 00000020:00080000:0.0:1787784774.773168:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 20 committed batchid 18 00000020:00080000:0.0:1787784774.773414:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 18 00000020:00080000:1.0:1787784774.773520:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 19 batchid updated 00000001:00080000:1.0:1787784774.773530:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968002 is committed 00000001:00080000:1.0:1787784774.773536:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968001 is committed 00010000:00010000:1.0:1787784775.607142:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787784776.427118:0:20287:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12n: Import/implicit restore/release == 18:52:54 (1787784774) 00010000:00010000:1.0:1787784776.625741:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784776.769948:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784776.772851:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658137088 00000100:00100000:1.0:1787784776.772867:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658137088 00000100:00100000:1.0:1787784776.772954:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1079 00000100:00100000:1.0:1787784776.772967:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28706:x1874627658137088:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1787784776.773085:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28706:x1874627658137088:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 121us (3146us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784776.773094:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1079 00000100:00080000:3.0:1787784776.773602:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784776.794835:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784776.794995:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658137216 00000100:00100000:1.0:1787784776.795004:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658137216 00000100:00100000:1.0:1787784776.795076:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1080 00000100:00100000:1.0:1787784776.795087:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:2356:x1874627658137216:12345-192.168.203.43@tcp:41:mkdir.0 00000004:00000024:1.0:1787784776.795172:0:10850:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 2259419136 avail 2257321984 00000100:00100000:1.0:1787784776.795249:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:2356:x1874627658137216:12345-192.168.203.43@tcp:41:mkdir.0 Request processed in 164us (422us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784776.795266:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1080 00000100:00080000:3.0:1787784776.795488:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784776.797612:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784776.798348:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658137472 00000100:00100000:1.0:1787784776.798358:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658137472 00000100:00100000:1.0:1787784776.798420:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1081 00000100:00100000:1.0:1787784776.798430:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28707:x1874627658137472:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:1.0:1787784776.798485:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784776.798510:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc5f8a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f81 expref: 91 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784776.798530:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407500) 00010000:00010000:1.0:1787784776.798534:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784776.798814:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784776.798828:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 41 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784776.798837:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784776.798868:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 41 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784776.798895:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 41 pid: 5812 timeout: 667 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784776.798923:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668403200:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784776.799312:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405ddc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787784776.818200:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784776.818300:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668403200:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784776.818327:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784776.818969:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658137600 00000100:00100000:3.0:1787784776.818977:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658137600 00010000:00010000:3.0:1787784776.819072:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 42 pid: 5812 timeout: 667 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784776.819108:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 328 00000100:00100000:3.0:1787784776.819116:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:2356:x1874627658137600:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784776.819131:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784776.819138:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 42 pid: 5812 timeout: 667 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784776.819178:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 42 pid: 5812 timeout: 667 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784776.819214:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc5fc9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0f96 expref: 42 pid: 5812 timeout: 667 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784776.819228:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407500) 00010000:00010000:3.0:1787784776.819233:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784776.819242:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784776.819249:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784776.819275:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784776.819313:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:2356:x1874627658137600:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 197us (1131us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784776.819321:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 328 00010000:00010000:2.0:1787784776.820139:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.820176:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.820207:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.820218:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.820227:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784776.823080:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 512 00010000:00010000:2.0:1787784776.823098:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.823109:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.823125:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.823132:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784776.823136:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.823165:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.823182:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407500) 00010000:00010000:2.0:1787784776.823185:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784776.823190:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784776.823197:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6223 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784776.823216:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a4188795 reply state 00000000e0ce1d73 transno 4294968003 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784776.823225:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784776.823315:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28707:x1874627658137472:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 24882us (25705us total) trans 4294968003 rc 0/0 00000100:00100000:2.0:1787784776.823324:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1081 00010000:00010000:0.0:1787784776.835804:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784776.835823:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784776.835831:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784776.835838:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784776.835843:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784776.835851:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784776.835890:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff989530cb1900) 00010000:00010000:0.0:1787784776.835893:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784776.835898:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784776.835906:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc622a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784777.202098:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784777.202122:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668403328:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784777.202210:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784777.202393:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668403328 00000100:00100000:2.0:1787784777.202402:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668403328 00000100:00100000:2.0:1787784777.202528:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1787784777.202539:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668403328:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784777.202591:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784777.202660:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668403328:12345-0@lo:13:osp-pre-0-0.0 Request processed in 123us (450us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784777.202667:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:1.0:1787784777.205867:0:3278:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967353 gen 2 00000004:00080000:1.0:1787784777.205880:0:3278:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967353 committed transno 4294967353 00000100:00100000:1.0:1787784777.205919:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668403328:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787784777.206355:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784777.297956:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.298056:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658137728 00000100:00100000:1.0:1787784777.298066:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658137728 00000100:00100000:1.0:1787784777.298130:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1082 00000100:00100000:1.0:1787784777.298141:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28744:x1874627658137728:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:1.0:1787784777.298199:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784777.298223:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6231 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1068 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.298290:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6238 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.298306:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6238 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.298319:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6238 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784777.298391:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784777.298395:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 28 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784777.298419:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6238 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.298445:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6231 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1068 expref: 93 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.298465:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6238 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1068 expref: 92 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.298474:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407700) 00010000:00010000:1.0:1787784777.298477:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784777.298480:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784777.298655:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28744:x1874627658137728:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 505us (699us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784777.298663:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1082 00000100:00100000:2.0:1787784777.322829:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.325123:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658137984 00000100:00100000:1.0:1787784777.325137:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658137984 00000100:00100000:1.0:1787784777.325287:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1083 00000100:00100000:1.0:1787784777.325298:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28744:x1874627658137984:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:1.0:1787784777.325352:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784777.325379:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6238 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1068 expref: 92 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.325401:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407700) 00010000:00010000:1.0:1787784777.325428:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784777.325600:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28744:x1874627658137984:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 302us (2779us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787784777.325609:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1083 00000100:00100000:2.0:1787784777.336453:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784777.337367:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658138112 00000100:00100000:0.0:1787784777.337378:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658138112 00000100:00100000:0.0:1787784777.337430:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1084 00000100:00100000:0.0:1787784777.337441:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+91:28744:x1874627658138112:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784777.337460:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784777.337483:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc623f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b106f expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.337547:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6246 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.337560:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6246 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.337570:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6246 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784777.337665:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784777.337669:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 28 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784777.337697:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6246 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.337721:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc623f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b106f expref: 93 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.337739:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6246 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b106f expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.337748:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784777.337751:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784777.337754:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:0.0:1787784777.337817:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28744:x1874627658138112:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 378us (1374us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784777.337824:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1084 00000100:00100000:0.0:1787784777.342364:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784777.342754:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658138240 00000100:00100000:2.0:1787784777.342763:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658138240 00000100:00100000:2.0:1787784777.342810:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1085 00000100:00100000:2.0:1787784777.342820:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+92:28744:x1874627658138240:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:2.0:1787784777.342839:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784777.342860:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc624d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1076 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.342919:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.342932:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.342943:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.342956:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.342961:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.342967:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343083:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6262 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343092:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6262 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343099:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6262 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784777.343136:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 512 00000004:00200000:2.0:1787784777.343180:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784777.343204:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6262 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343216:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343222:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343379:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343384:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784777.343388:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343396:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343408:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:2.0:1787784777.343416:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784777.343421:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343428:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6254 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343442:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343449:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343455:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343460:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784777.343463:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343468:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343703:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff989530cb1c00) 00010000:00010000:2.0:1787784777.343708:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784777.343713:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343719:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc625b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343750:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc624d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1076 expref: 94 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343774:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6262 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1076 expref: 93 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.343780:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784777.343782:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784777.343785:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784777.343851:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28744:x1874627658138240:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 1032us (1495us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784777.343858:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1085 00000100:00100000:2.0:1787784777.445211:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784777.445473:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658138368 00000100:00100000:0.0:1787784777.445483:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658138368 00000100:00100000:0.0:1787784777.445589:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1086 00000100:00100000:0.0:1787784777.445601:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+41:28750:x1874627658138368:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784777.445620:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784777.445656:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6269 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b107d expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.445719:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.445744:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.445760:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784777.445828:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784777.445832:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 28 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784777.445857:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.445881:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6269 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 43 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.445898:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.445906:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784777.445909:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784777.445913:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031800, rc 0) 00000100:00100000:0.0:1787784777.445974:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28750:x1874627658138368:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 376us (773us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784777.445981:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1086 00000100:00100000:0.0:1787784777.453258:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784777.453345:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658138624 00000100:00100000:2.0:1787784777.453354:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658138624 00000100:00100000:2.0:1787784777.453415:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1087 00000100:00100000:2.0:1787784777.453427:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28750:x1874627658138624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784777.453449:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784777.453452:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787784777.453479:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6262 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1076 expref: 93 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.453497:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff989530cb1500) 00010000:00010000:2.0:1787784777.453510:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784777.453524:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6277 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1084 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.453666:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.453692:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.453706:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.453754:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.453762:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.453769:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454379:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454390:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454397:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784777.454406:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 0 00010000:00010000:2.0:1787784777.454430:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454438:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454444:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454457:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784777.454461:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454469:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454479:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:2.0:1787784777.454482:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784777.454485:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454491:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc627e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784777.454501:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a830b39c reply state 0000000098b3ddc7 transno 4294968004 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784777.454509:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784777.454537:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6277 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 95 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454554:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 94 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.454561:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:2.0:1787784777.454564:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784777.454566:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:2.0:1787784777.454659:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:28750:x1874627658138624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1234us (1410us total) trans 4294968004 rc 0/0 00000100:00100000:2.0:1787784777.454673:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1087 00010000:00010000:0.0:1787784777.458335:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.458352:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.458379:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.458435:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784777.458440:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.458448:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.458464:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x81b47e8f (ffff989530cb1900) 00010000:00010000:0.0:1787784777.458467:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784777.458532:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.458539:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6285 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784777.461376:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.461450:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658138880 00000100:00100000:1.0:1787784777.461460:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658138880 00000100:00100000:1.0:1787784777.461520:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1088 00000100:00100000:1.0:1787784777.461531:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:28750:x1874627658138880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784777.461553:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784777.461575:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc629a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b108b expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.461685:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.461699:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 95 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.461708:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.461740:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 95 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.461766:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 95 pid: 5812 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784777.461780:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668403456:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784777.462621:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405ddc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784777.471565:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784777.471667:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658139008 00000100:00100000:0.0:1787784777.471675:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658139008 00010000:00010000:0.0:1787784777.471716:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 96 pid: 5812 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784777.471765:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:0.0:1787784777.471776:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2357:x1874627658139008:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784777.471793:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784777.471800:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 96 pid: 5812 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.471826:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 96 pid: 5812 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.471848:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:0.0:1787784777.471854:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.471864:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.471881:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.471901:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784777.472053:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2357:x1874627658139008:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 278us (496us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784777.472071:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 329 00010000:00010000:1.0:1787784777.474461:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc628c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1084 expref: 95 pid: 5812 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784777.474487:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668403456:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784777.474510:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.474520:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.474526:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784777.474548:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 0 00010000:00010000:1.0:1787784777.475839:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.475853:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.475860:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.475865:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784777.475869:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.475997:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.476029:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1e00) 00010000:00010000:1.0:1787784777.476032:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784777.476036:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.476041:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62a1 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.476409:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc629a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b108b expref: 95 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.476421:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff98951c407700) 00010000:00010000:1.0:1787784777.476424:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784777.476427:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc629a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b108b expref: 95 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.476439:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784777.476553:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:28750:x1874627658138880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 15023us (15185us total) trans 4294968005 rc 301/301 00000100:00100000:1.0:1787784777.476563:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1088 00000100:00100000:1.0:1787784777.486388:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.486461:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.486543:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.486571:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784777.486609:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658139136 00000100:00100000:0.0:1787784777.486618:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658139136 00000100:00100000:0.0:1787784777.486682:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 557 00000100:00100000:0.0:1787784777.486693:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658139136:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784777.486765:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658139136:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 74us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784777.486773:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 557 00000100:00100000:3.0:1787784777.487078:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658139264 00000100:00100000:3.0:1787784777.487090:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658139264 00000100:00100000:3.0:1787784777.487136:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658139520 00000100:00100000:3.0:1787784777.487139:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658139520 00000100:00100000:3.0:1787784777.487174:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:3.0:1787784777.487181:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658139264:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784777.487220:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658139264:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (841us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784777.487233:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:3.0:1787784777.487260:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 336 00000100:00100000:3.0:1787784777.487265:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658139520:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784777.487291:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658139520:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (830us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784777.487306:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 336 00000100:00080000:0.0:1787784777.487553:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784777.490359:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.490415:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784777.490442:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658139392 00000100:00100000:1.0:1787784777.490445:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658139904 00000100:00100000:0.0:1787784777.490451:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658139392 00000100:00100000:1.0:1787784777.490457:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658139904 00000100:00100000:1.0:1787784777.490493:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 216 00000100:00100000:1.0:1787784777.490503:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d7850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:28750:x1874627658139904:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784777.490505:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658139648 00000100:00100000:0.0:1787784777.490511:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658139648 00000100:00100000:0.0:1787784777.490533:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:0.0:1787784777.490540:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658139392:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784777.490602:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658139392:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 61us (4029us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784777.490609:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:0.0:1787784777.490631:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 338 00000100:00100000:0.0:1787784777.490654:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658139648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784777.490683:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658139648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 45us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784777.490690:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 338 00000100:00080000:0.0:1787784777.490709:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784777.490797:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d7850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:28750:x1874627658139904:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 291us (375us total) trans 4294968006 rc 0/0 00000100:00100000:1.0:1787784777.490804:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 216 00000100:00100000:3.0:1787784777.494258:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784777.494776:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658140032 00000100:00100000:2.0:1787784777.494785:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658140032 00000100:00100000:2.0:1787784777.494844:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1089 00000100:00100000:2.0:1787784777.494852:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:28750:x1874627658140032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784777.494882:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784777.494904:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc62af lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1092 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.494965:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc62af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1092 expref: 96 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.494978:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc62af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1092 expref: 96 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784777.494995:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:2.0:1787784777.494998:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784777.495002:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784777.495057:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:28750:x1874627658140032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 205us (807us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784777.495065:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1089 00000100:00100000:1.0:1787784777.505571:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784777.505679:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658140288 00000100:00100000:0.0:1787784777.505689:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658140288 00000100:00100000:0.0:1787784777.505746:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1090 00000100:00100000:0.0:1787784777.505755:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:28750:x1874627658140288:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1787784777.505817:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.505831:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.505842:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.506041:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.506061:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.506068:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.506073:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784777.506077:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.506084:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.506096:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:0.0:1787784777.506099:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787784777.506103:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.506109:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc62b6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784777.506347:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:28750:x1874627658140288:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 591us (783us total) trans 4294968007 rc 0/0 00000100:00100000:0.0:1787784777.506354:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1090 00000100:00100000:0.0:1787784777.515966:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.516410:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658140544 00000100:00100000:1.0:1787784777.516420:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658140544 00000100:00100000:1.0:1787784777.516483:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1091 00000100:00100000:1.0:1787784777.516493:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:28750:x1874627658140544:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784777.516771:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.516789:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.516799:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784777.516864:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968008 00000004:00010000:1.0:1787784777.516901:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003af1589d reply state 0000000018c4e91e transno 4294968009 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784777.516908:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784777.517248:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:28750:x1874627658140544:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 755us (1290us total) trans 4294968009 rc 0/0 00000100:00100000:1.0:1787784777.517256:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1091 00000020:00080000:1.0:1787784777.517261:0:5811:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1787784777.524858:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.524922:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658140800 00000100:00100000:1.0:1787784777.524931:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658140800 00000100:00100000:1.0:1787784777.524984:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 217 00000100:00100000:1.0:1787784777.524994:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:28750:x1874627658140800:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784777.525228:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28750:x1874627658140800:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 235us (378us total) trans 4294968010 rc 0/0 00000100:00100000:1.0:1787784777.525236:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 217 00010000:00010000:0.0:1787784777.525619:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.525629:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.525663:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.525670:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784777.525675:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.525682:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.525697:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:0.0:1787784777.525700:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784777.525705:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 2/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784777.525711:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc62bd lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784777.585494:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784777.585503:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784777.586026:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1787784777.586035:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784777.649748:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784777.672944:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784777.673351:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658140928 00000100:00100000:1.0:1787784777.673362:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658140928 00000100:00100000:1.0:1787784777.673508:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1092 00000100:00100000:1.0:1787784777.673518:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:28756:x1874627658140928:12345-192.168.203.43@tcp:101:cmp.0 00010000:00010000:1.0:1787784777.673540:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784777.673559:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc62c4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1099 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673621:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673634:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673655:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673670:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673678:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673703:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673807:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc62d9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673817:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc62d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673825:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc62d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784777.673863:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 512 00000004:00200000:1.0:1787784777.673868:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784777.673893:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc62d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673906:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673913:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673933:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673940:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784777.673944:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673952:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673971:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:1.0:1787784777.673974:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784777.673979:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673986:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc62cb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.673999:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674005:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674011:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674017:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784777.674020:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674026:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674034:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff98951c407700) 00010000:00010000:1.0:1787784777.674037:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784777.674040:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674046:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc62d2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674087:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc62c4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1099 expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674100:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c400/0xb0ce63137acc62d9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1099 expref: 101 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784777.674108:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff98951c407900) 00010000:00010000:1.0:1787784777.674110:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784777.674113:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c400, rc 0) 00000100:00100000:1.0:1787784777.674232:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28756:x1874627658140928:12345-192.168.203.43@tcp:101:cmp.0 Request processed in 714us (1297us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784777.674239:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1092 00000100:00100000:2.0:1787784777.679628:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784777.680463:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658141184 00000100:00100000:3.0:1787784777.680474:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658141184 00000100:00100000:3.0:1787784777.680535:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1093 00000100:00100000:3.0:1787784777.680545:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28756:x1874627658141184:12345-192.168.203.43@tcp:101:cmp.0 00010000:00010000:3.0:1787784777.680565:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784777.680585:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc62e0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10a0 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680672:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680687:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680699:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680713:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680720:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680745:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680864:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc62f5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680873:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc62f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.680879:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc62f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784777.680888:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00010000:00010000:3.0:1787784777.680993:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681001:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681008:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681013:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784777.681018:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681026:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681037:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff98951c407900) 00010000:00010000:3.0:1787784777.681040:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784777.681051:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681209:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc62e7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681337:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681345:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681351:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681356:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784777.681359:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681365:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681373:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x81b47e8f (ffff989505542e00) 00010000:00010000:3.0:1787784777.681375:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784777.681378:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 2/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681385:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc62ee lrc: 0/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681424:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc62e0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a0 expref: 104 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681449:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc62f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a0 expref: 103 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.681457:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:3.0:1787784777.681460:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784777.681462:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7400, rc 0) 00000100:00100000:3.0:1787784777.681512:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:28756:x1874627658141184:12345-192.168.203.43@tcp:101:cmp.0 Request processed in 969us (1892us total) trans 4294968011 rc 0/0 00000100:00100000:3.0:1787784777.681519:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1093 00000100:00100000:2.0:1787784777.689838:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784777.689927:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658141440 00000100:00100000:3.0:1787784777.689937:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658141440 00000100:00100000:3.0:1787784777.690000:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 47 00000100:00100000:3.0:1787784777.690011:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:28756:x1874627658141440:12345-192.168.203.43@tcp:103:cmp.0 00010000:00010000:3.0:1787784777.690025:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1787784777.690048:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc62f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a0 expref: 103 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.690078:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc62af lrc: 0/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1092 expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.690089:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989530cb1c00) 00010000:00010000:3.0:1787784777.690099:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784777.690383:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28756:x1874627658141440:12345-192.168.203.43@tcp:103:cmp.0 Request processed in 374us (553us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784777.690390:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 47 00000100:00100000:0.0:1787784777.708052:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784777.708195:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658141568 00000100:00100000:3.0:1787784777.708205:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658141568 00000100:00100000:3.0:1787784777.708273:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1094 00000100:00100000:3.0:1787784777.708284:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28756:x1874627658141568:12345-192.168.203.43@tcp:58:cmp.0 00010000:00010000:3.0:1787784777.708379:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.708397:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.708407:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784777.708602:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].30, 136 off12272 00000100:00100000:3.0:1787784777.708806:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28756:x1874627658141568:12345-192.168.203.43@tcp:58:cmp.0 Request processed in 523us (762us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784777.708815:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1094 00000100:00100000:2.0:1787784777.713942:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784777.714021:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658141696 00000100:00100000:3.0:1787784777.714038:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658141696 00000100:00100000:3.0:1787784777.714101:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1095 00000100:00100000:3.0:1787784777.714111:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:28756:x1874627658141696:12345-192.168.203.43@tcp:101:cmp.0 00010000:00010000:3.0:1787784777.714131:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784777.714181:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10a7 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.714446:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.714460:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 4/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.714493:0:5811:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.714507:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 3/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.714521:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:3.0:1787784777.714524:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784777.714528:0:5811:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 3/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784777.714536:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6800, rc 0) 00000100:00100000:3.0:1787784777.714583:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:28756:x1874627658141696:12345-192.168.203.43@tcp:101:cmp.0 Request processed in 473us (649us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784777.714590:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1095 00000040:00080000:2.0:1787784778.609531:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784778.609541:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784778.609581:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 31 count 4 00000040:00080000:2.0:1787784778.609588:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784778.611182:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668403584:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784778.611293:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a59c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784778.614491:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668403584:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784778.615953:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.616049:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658141824 00000100:00100000:2.0:1787784778.616056:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658141824 00000100:00100000:2.0:1787784778.616123:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1096 00000100:00100000:2.0:1787784778.616134:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28757:x1874627658141824:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784778.616192:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x44:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784778.616309:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28757:x1874627658141824:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 177us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784778.616317:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1096 00000100:00080000:0.0:1787784778.616471:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784778.624816:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.624901:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658141952 00000100:00100000:3.0:1787784778.624910:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658141952 00000100:00100000:3.0:1787784778.624972:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1097 00000100:00100000:3.0:1787784778.624984:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28757:x1874627658141952:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1787784778.625057:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625075:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625089:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625123:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625131:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625138:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625188:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625197:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625204:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784778.625250:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00000004:00200000:3.0:1787784778.625255:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784778.625268:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625279:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625285:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625292:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625309:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784778.625313:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625321:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625333:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784778.625337:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784778.625341:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625348:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6311 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625361:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625367:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625372:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625377:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784778.625380:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625386:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625404:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x8b3c9e9 (ffff989505542f00) 00010000:00010000:3.0:1787784778.625406:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784778.625409:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625415:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6318 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625428:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625435:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625440:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625445:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784778.625448:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625454:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625461:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:3.0:1787784778.625463:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784778.625466:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.625472:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc631f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.625556:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28757:x1874627658141952:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 573us (747us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.625563:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1097 00000100:00100000:1.0:1787784778.630483:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.633867:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658142208 00000100:00100000:2.0:1787784778.633882:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658142208 00000100:00100000:2.0:1787784778.633947:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1098 00000100:00100000:2.0:1787784778.633958:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+42:28757:x1874627658142208:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784778.633979:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784778.634001:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc6326 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10ae expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634118:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634133:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634166:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784778.634178:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00010000:00010000:2.0:1787784778.634289:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634298:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634305:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634311:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784778.634316:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634323:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634335:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:2.0:1787784778.634339:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784778.634343:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634355:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc632d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634393:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc6326 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ae expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634403:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784778.634406:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784778.634409:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc6326 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ae expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.634419:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784778.634482:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:28757:x1874627658142208:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 525us (4008us total) trans 4294968012 rc 301/301 00000100:00100000:2.0:1787784778.634490:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1098 00000100:00100000:3.0:1787784778.637840:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.637911:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658142336 00000100:00100000:1.0:1787784778.637920:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658142336 00000100:00100000:1.0:1787784778.637976:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1099 00000100:00100000:1.0:1787784778.637995:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:28757:x1874627658142336:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784778.638052:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x44:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784778.638191:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:28757:x1874627658142336:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 204us (364us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.638199:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1099 00000100:00100000:1.0:1787784778.649759:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.649841:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658142464 00000100:00100000:2.0:1787784778.649850:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658142464 00000100:00100000:2.0:1787784778.649904:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1100 00000100:00100000:2.0:1787784778.649914:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:28757:x1874627658142464:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784778.649935:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784778.649957:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc633b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10b5 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650017:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650029:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650041:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650055:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650061:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650067:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650520:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6350 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650530:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6350 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650538:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6350 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784778.650566:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 512 00000004:00200000:2.0:1787784778.650570:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784778.650591:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6350 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650603:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650610:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650621:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650626:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784778.650630:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650650:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650662:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:2.0:1787784778.650664:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784778.650669:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650675:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6342 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650688:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650694:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650700:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650705:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784778.650708:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650718:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650724:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff989530cb1500) 00010000:00010000:2.0:1787784778.650726:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784778.650729:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650735:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6349 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650765:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc633b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10b5 expref: 45 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650777:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd600/0xb0ce63137acc6350 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10b5 expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.650783:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff98951c407900) 00010000:00010000:2.0:1787784778.650785:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784778.650787:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd600, rc 0) 00000100:00100000:2.0:1787784778.650846:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28757:x1874627658142464:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 934us (1098us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784778.650854:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1100 00000100:00100000:3.0:1787784778.658309:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.658412:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658142720 00000100:00100000:1.0:1787784778.658422:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658142720 00000100:00100000:1.0:1787784778.658504:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1101 00000100:00100000:1.0:1787784778.658513:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:28757:x1874627658142720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784778.658532:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784778.658552:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6357 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10bc expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787784778.659840:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x18:0x0], index 1: 0 00010000:00010000:1.0:1787784778.660376:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc635e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.660392:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc635e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.660404:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc635e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.660476:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6357 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10bc expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.660494:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc635e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10bc expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.660503:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407700) 00010000:00010000:1.0:1787784778.660506:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784778.660509:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:1.0:1787784778.660578:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28757:x1874627658142720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2066us (2278us total) trans 4294968013 rc 0/0 00000100:00100000:1.0:1787784778.660588:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1101 00000100:00100000:3.0:1787784778.664048:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.664230:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658142976 00000100:00100000:1.0:1787784778.664253:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658142976 00000100:00100000:1.0:1787784778.664321:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1102 00000100:00100000:1.0:1787784778.664333:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28757:x1874627658142976:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784778.665496:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784778.665525:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc635e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10bc expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.665544:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407700) 00010000:00010000:1.0:1787784778.665554:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784778.665575:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.665590:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.665600:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.665625:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.665936:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.665946:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666013:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666021:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666039:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666045:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784778.666049:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666058:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666067:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff98951c407900) 00010000:00010000:1.0:1787784778.666070:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784778.666074:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666081:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc636c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666092:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666098:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666103:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666108:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784778.666111:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666127:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666136:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x635f7999 (ffff98951c407300) 00010000:00010000:1.0:1787784778.666138:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784778.666141:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.666196:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6373 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x635f7999 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.666280:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28757:x1874627658142976:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 1948us (2240us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784778.666289:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1102 00000100:00100000:2.0:1787784778.670923:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.671005:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658143104 00000100:00100000:1.0:1787784778.671013:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658143104 00000100:00100000:1.0:1787784778.671066:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1103 00000100:00100000:1.0:1787784778.671075:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28757:x1874627658143104:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784778.671094:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784778.671116:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc637a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10c3 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.671542:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.671557:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.671571:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.672004:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc637a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 47 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.672025:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.672034:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:1.0:1787784778.672037:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784778.672040:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784778.672102:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28757:x1874627658143104:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1027us (1187us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.672110:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1103 00010000:00010000:0.0:1787784778.674117:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784778.682362:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.682587:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658143360 00000100:00100000:2.0:1787784778.682606:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658143360 00000100:00100000:2.0:1787784778.682714:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1104 00000100:00100000:2.0:1787784778.682723:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28757:x1874627658143360:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784778.682804:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.682820:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.682832:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784778.683095:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].15, 88 off9248 00000004:00080000:2.0:1787784778.683105:0:5812:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 569769957440ns, next at 570769954449ns, rc = 0 00000004:00010000:2.0:1787784778.683482:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007e169b5f reply state 000000009a27295e transno 4294968014 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784778.683492:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784778.683606:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28757:x1874627658143360:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 872us (1240us total) trans 4294968014 rc 0/0 00000100:00100000:2.0:1787784778.683622:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1104 00010000:00010000:0.0:1787784778.690848:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.690864:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.690872:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.690894:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784778.690899:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.690908:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.690924:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:0.0:1787784778.690927:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784778.690932:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.690938:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6388 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.698760:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.699080:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658143488 00000100:00100000:1.0:1787784778.699090:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658143488 00000100:00100000:1.0:1787784778.699184:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1105 00000100:00100000:1.0:1787784778.699195:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28757:x1874627658143488:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784778.699218:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784778.699238:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10ca expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.699296:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.699310:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.699349:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:1.0:1787784778.699353:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784778.699356:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784778.699414:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28757:x1874627658143488:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 221us (663us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.699421:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1105 00000100:00100000:3.0:1787784778.704242:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784778.704883:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658143616 00000100:00100000:0.0:1787784778.704893:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658143616 00000100:00100000:0.0:1787784778.704949:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 292 00000100:00100000:0.0:1787784778.704959:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658143616:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1787784778.705599:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658143616:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 638us (1363us total) trans 4294967360 rc 0/0 00000100:00100000:0.0:1787784778.705608:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 292 00000100:00100000:3.0:1787784778.710211:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.710399:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658143872 00000100:00100000:1.0:1787784778.710409:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658143872 00000100:00100000:1.0:1787784778.710470:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 218 00000100:00100000:1.0:1787784778.710480:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:28757:x1874627658143872:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784778.711171:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28757:x1874627658143872:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 668us (945us total) trans 4294968015 rc 0/0 00000100:00100000:1.0:1787784778.711181:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 218 00000100:00100000:1.0:1787784778.714688:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.714875:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658144128 00000100:00100000:3.0:1787784778.714885:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658144128 00000100:00100000:3.0:1787784778.714989:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1106 00000100:00100000:3.0:1787784778.715000:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28757:x1874627658144128:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784778.715116:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28757:x1874627658144128:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 117us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.715123:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1106 00000100:00100000:2.0:1787784778.719312:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.719406:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658144256 00000100:00100000:1.0:1787784778.719415:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658144256 00000100:00100000:1.0:1787784778.719569:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1107 00000100:00100000:1.0:1787784778.719580:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28757:x1874627658144256:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784778.719601:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784778.719652:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6396 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10d1 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.719737:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc639d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.719769:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc639d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.719780:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc639d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784778.719817:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784778.719836:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc639d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.719862:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6396 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d1 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.719881:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc639d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d1 expref: 50 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.719889:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:1.0:1787784778.719893:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1787784778.719896:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:1.0:1787784778.719959:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:28757:x1874627658144256:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 381us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.719966:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1107 00000100:00100000:0.0:1787784778.723245:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.723321:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658144384 00000100:00100000:3.0:1787784778.723328:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658144384 00000100:00100000:3.0:1787784778.723386:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 293 00000100:00100000:3.0:1787784778.723396:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035970c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658144384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784778.723414:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784778.723439:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10d8 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784778.723501:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x18:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a8f6e4a, blocks: 0x1 00010000:00010000:3.0:1787784778.723523:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=569s v=(0 0) 00010000:00010000:3.0:1787784778.723541:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784778.723551:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.723568:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.723582:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784778.723659:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035970c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658144384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 264us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.723666:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 293 00000100:00100000:0.0:1787784778.727692:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.727782:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658144640 00000100:00100000:2.0:1787784778.727792:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658144640 00000100:00100000:2.0:1787784778.727850:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1108 00000100:00100000:2.0:1787784778.727860:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:28757:x1874627658144640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784778.727965:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:28757:x1874627658144640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 105us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784778.727972:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1108 00000100:00100000:2.0:1787784778.734546:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.734795:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658144768 00000100:00100000:1.0:1787784778.734804:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658144768 00000100:00100000:1.0:1787784778.734908:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 294 00000100:00100000:1.0:1787784778.734918:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:28757:x1874627658144768:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784778.734936:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784778.734966:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 2/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10df expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.734999:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.735008:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784778.735013:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784778.735018:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 4/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.735239:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.735263:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 13 pid: 6669 timeout: 669 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.735279:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668403712:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784778.735374:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784778.748768:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.750270:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658144896 00000100:00100000:3.0:1787784778.750282:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658144896 00010000:00010000:3.0:1787784778.750325:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 14 pid: 6669 timeout: 669 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.750369:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:3.0:1787784778.750379:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403597480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658144896:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784778.750395:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784778.750414:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 14 pid: 6669 timeout: 669 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.750429:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 14 pid: 6669 timeout: 669 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.750454:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784778.750471:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784778.750479:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 14 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.750491:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 14 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.750529:0:11303:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 14 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.750543:0:11303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11303:1874627668403840:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784778.750629:0:11303:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508835900 going to sleep for 11 seconds 00010000:00010000:1.0:1787784778.750737:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc63a4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d8 expref: 14 pid: 6669 timeout: 669 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.750753:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668403712:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784778.750776:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.750800:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c600, rc 0) 00000100:00100000:1.0:1787784778.750828:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+13:28757:x1874627658144768:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 15912us (16291us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.750837:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 294 00000100:00100000:3.0:1787784778.755119:0:11303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11303:1874627668403840:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784778.755143:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784778.755234:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403597480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658144896:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 4848us (6466us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.755246:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:0.0:1787784778.756612:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.756687:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658145024 00000100:00100000:3.0:1787784778.756693:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658145024 00002000:00100000:3.0:1787784778.756736:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x18:0x0] (0->157) req@ffff9894035952c0 x1874627658145024/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:659/0 lens 648/0 e 0 to 0 dl 1787784789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784778.756747:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894035952c0 with x1874627658145024 ext(0->157) 00002000:00010000:3.0:1787784778.756752:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894035952c0 00000100:00100000:3.0:1787784778.756778:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 99 00000100:00100000:3.0:1787784778.756786:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035952c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658145024:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1787784778.757312:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035952c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658145024:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 524us (703us total) trans 4294967361 rc 0/0 00000100:00100000:3.0:1787784778.757320:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 99 00002000:00100000:3.0:1787784778.757327:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x18:0x0] (0->157) req@ffff9894035952c0 x1874627658145024/t4294967361(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:659/0 lens 648/448 e 0 to 0 dl 1787784789 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784778.757336:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894035952c0 with x1874627658145024 ext(0->157) 00002000:00010000:3.0:1787784778.757340:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894035952c0 00000100:00100000:0.0:1787784778.758898:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.758958:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658145152 00000100:00100000:2.0:1787784778.758967:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658145152 00000100:00100000:2.0:1787784778.759066:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1109 00000100:00100000:2.0:1787784778.759073:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035943c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:28757:x1874627658145152:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:0.0:1787784778.792685:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968007 is committed 00000020:00080000:0.0:1787784778.792703:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 20 batchid updated 00000001:00080000:0.0:1787784778.792720:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968015 is committed 00000001:00080000:0.0:1787784778.792731:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968010 is committed 00000004:00080000:0.0:1787784778.792745:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 16 committed 00000001:00080000:0.0:1787784778.792766:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968011 is committed 00000020:00080000:3.0:1787784778.793009:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 20 00000020:00080000:3.0:1787784778.793023:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 20 00000100:00100000:1.0:1787784778.793130:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784778.793144:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035943c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28757:x1874627658145152:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 34065us (34246us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.793171:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668403968:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1787784778.793238:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784778.793380:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1109 00000100:00100000:1.0:1787784778.793546:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668403968 00000100:00100000:1.0:1787784778.793553:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668403968 00000100:00080000:0.0:1787784778.793605:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784778.793607:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1787784778.793615:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668403968:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1787784778.793782:0:6671:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:1.0:1787784778.794218:0:6671:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb mtime from disk: 0 -> 1787784778 00002000:00010000:1.0:1787784778.794222:0:6671:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1787784778.794224:0:6671:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb ctime from disk: 1787784778 -> 1787784778 00002000:00010000:1.0:1787784778.794227:0:6671:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:0.0:1787784778.794511:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668403968:12345-0@lo:2:osp-syn-1-0.0 Request processed in 895us (1272us total) trans 4294967362 rc 0/0 00000100:00100000:0.0:1787784778.794520:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000004:00080000:1.0:1787784778.794545:0:3277:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff98953f7b1680/2, rc 0, transno 4294967362 00000100:00100000:1.0:1787784778.794553:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668403968:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1787784778.795801:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.796020:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658145280 00000100:00100000:3.0:1787784778.796028:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658145280 00000100:00100000:3.0:1787784778.796070:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00100000:3.0:1787784778.796076:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658145280:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:0.0:1787784778.838446:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967360 is committed 00000001:00080000:0.0:1787784778.838534:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967362 is committed 00000001:00080000:0.0:1787784778.838624:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967361 is committed 00000100:00100000:3.0:1787784778.838704:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658145280:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 42621us (42901us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.838717:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00100000:2.0:1787784778.841365:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.841507:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658145536 00000100:00100000:1.0:1787784778.841516:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658145536 00000100:00100000:1.0:1787784778.841570:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1110 00000100:00100000:1.0:1787784778.841580:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28757:x1874627658145536:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784778.841877:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784778.841903:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc639d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10d1 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.841933:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:1.0:1787784778.841938:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784778.841967:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.841979:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.841988:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784778.842198:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fa324462 reply state 00000000c95b4723 transno 4294968016 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.842208:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784778.842467:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:28757:x1874627658145536:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 887us (1108us total) trans 4294968016 rc 0/0 00000100:00100000:1.0:1787784778.842475:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1110 00010000:00010000:0.0:1787784778.844136:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.844187:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.844194:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.844200:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784778.844205:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.844213:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.844236:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:0.0:1787784778.844239:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784778.844257:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.844268:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc63b2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784778.845867:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.846800:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658145664 00000100:00100000:1.0:1787784778.846809:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658145664 00000100:00100000:1.0:1787784778.846858:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 297 00000100:00100000:1.0:1787784778.846866:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658145664:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1787784778.847057:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:1.0:1787784778.847060:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb mtime from disk: 1787784778 -> 1536580263 00002000:00010000:1.0:1787784778.847062:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb atime from disk: 0 -> 1787784232 00002000:00010000:1.0:1787784778.847064:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb ctime from disk: 1787784778 -> 1787784776 00002000:00010000:1.0:1787784778.847066:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1787784778.847144:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658145664:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 277us (1284us total) trans 4294967363 rc 0/0 00000100:00100000:1.0:1787784778.847171:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 297 00000100:00100000:1.0:1787784778.850866:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.851142:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658145920 00000100:00100000:3.0:1787784778.851242:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658145920 00000100:00100000:2.0:1787784778.851879:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1111 00000100:00100000:2.0:1787784778.851888:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:28757:x1874627658145920:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1787784778.851956:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.851974:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.851983:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.852017:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.852030:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 46 pid: 5813 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784778.852043:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ad680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668404096:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784778.852139:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a53c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784778.854671:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.855141:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ad680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668404096:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784778.855192:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.855343:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658146048 00000100:00100000:3.0:1787784778.855351:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658146048 00010000:00010000:3.0:1787784778.855384:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 47 pid: 5813 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.855412:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00100000:3.0:1787784778.855420:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658146048:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784778.855448:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784778.855454:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 47 pid: 5813 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.855466:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 47 pid: 5813 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.855487:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6381 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10c3 expref: 47 pid: 5813 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.855508:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:3.0:1787784778.855512:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.855522:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.855529:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.855557:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784778.855595:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2356:x1874627658146048:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 175us (931us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.855602:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 331 00010000:00010000:0.0:1787784778.855862:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855871:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855895:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855902:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855908:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855914:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784778.855918:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855925:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855934:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:0.0:1787784778.855937:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787784778.855940:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.855946:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc63b9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.856006:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28757:x1874627658145920:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 4119us (5150us total) trans 0 rc -22/-22 00000100:00100000:0.0:1787784778.856014:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1111 00000100:00100000:2.0:1787784778.859118:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784778.859451:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658146176 00000100:00100000:0.0:1787784778.859459:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658146176 00000100:00100000:0.0:1787784778.859532:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 298 00000100:00100000:0.0:1787784778.859540:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658146176:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784778.859578:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.859596:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.859606:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784778.859610:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784778.859615:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.859659:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.859681:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 12 pid: 13825 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.859695:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668404224:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784778.859858:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895407279c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784778.863060:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.863454:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658146304 00000100:00100000:1.0:1787784778.863463:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658146304 00010000:00010000:1.0:1787784778.863501:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 13 pid: 13825 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.863537:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 332 00000100:00100000:1.0:1787784778.863546:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658146304:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784778.863559:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784778.863581:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 13 pid: 13825 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.863594:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 13 pid: 13825 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.863619:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784778.863623:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784778.863642:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.863919:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.863945:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784778.863999:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658146304:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 454us (948us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.864007:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 332 00010000:00010000:0.0:1787784778.865925:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc63ab lrc: 0/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10df expref: 12 pid: 13825 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.865947:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668404224:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784778.865969:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.865980:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.865987:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.866018:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.866026:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.866034:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.866040:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784778.866044:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.866073:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.866090:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.866097:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc63c0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.866416:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658146176:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 6874us (7304us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784778.866427:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 298 00000100:00100000:2.0:1787784778.869181:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.869372:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658146560 00000100:00100000:3.0:1787784778.869381:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658146560 00000100:00100000:0.0:1787784778.869527:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1112 00000100:00100000:0.0:1787784778.869539:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:28757:x1874627658146560:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784778.869612:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.869628:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.869654:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.869883:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.869910:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.869918:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.869949:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.869962:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 46 pid: 5811 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.869975:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668404352:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784778.870043:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895407279c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784778.873973:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.874048:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658146688 00000100:00100000:3.0:1787784778.874056:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658146688 00010000:00010000:3.0:1787784778.874091:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 47 pid: 5811 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.874179:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 333 00000100:00100000:3.0:1787784778.874190:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658146688:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784778.874207:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784778.874223:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 47 pid: 5811 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.874238:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 47 pid: 5811 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.874260:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:3.0:1787784778.874265:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.874274:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.874282:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.874310:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784778.874357:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658146688:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 169us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.874365:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 333 00010000:00010000:0.0:1787784778.875313:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc638f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ca expref: 46 pid: 5811 timeout: 669 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.875343:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668404352:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784778.875364:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.875374:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.875380:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784778.875724:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968017 00000001:00080000:0.0:1787784778.875891:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968018 00000001:00080000:0.0:1787784778.876020:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968019 00000100:00100000:0.0:1787784778.876100:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668404480:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784778.876141:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784778.876287:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895407276c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784778.876334:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668404480 00000100:00100000:3.0:1787784778.876342:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668404480 00000100:00100000:3.0:1787784778.876387:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 45 00000100:00100000:3.0:1787784778.876396:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403597c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668404480:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787784778.876480:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403597c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668404480:12345-0@lo:1000:mdt00_003.0 Request processed in 85us (337us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.876488:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 45 00000100:00100000:0.0:1787784778.876849:0:10850:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967362 gen 2 00000004:00080000:0.0:1787784778.876856:0:10850:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff98953f7b1680, transno 4294967362 00000100:00100000:0.0:1787784778.876876:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668404480:0@lo:1000:mdt00_003.0 00000001:00080000:0.0:1787784778.877331:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968020 00000100:00100000:0.0:1787784778.877389:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668404608:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784778.877429:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784778.877499:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506773f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784778.877603:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668404608 00000100:00100000:3.0:1787784778.877609:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668404608 00000100:00100000:3.0:1787784778.877654:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:3.0:1787784778.877664:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668404608:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787784778.877858:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:10850:x1874627668404608:12345-0@lo:1000:mdt00_003.0 Request processed in 195us (428us total) trans 4294967364 rc 0/0 00000100:00100000:3.0:1787784778.877866:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00100000:0.0:1787784778.877989:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668404608:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787784778.878020:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878030:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878038:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878044:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784778.878049:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878057:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878070:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff98951c407500) 00010000:00010000:0.0:1787784778.878081:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784778.878086:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878092:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc63ce lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878119:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878125:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878130:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878135:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784778.878138:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878144:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878261:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:0.0:1787784778.878264:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784778.878271:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 92 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878282:0:10850:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 92 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878292:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 92 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.878325:0:10850:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 92 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.878335:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668404736:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784778.878399:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895407279c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784778.882216:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668404736:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784778.882246:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882256:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6303 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882290:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882303:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882314:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882324:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882330:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882336:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882356:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784778.882360:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882367:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882378:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:0.0:1787784778.882380:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784778.882384:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882389:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc63d5 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882399:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882405:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882410:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882415:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784778.882417:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882423:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882441:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:0.0:1787784778.882443:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784778.882446:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.882452:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc63c7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.882779:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.882847:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658146816 00000100:00100000:3.0:1787784778.882853:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658146816 00000100:00100000:0.0:1787784778.882853:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28757:x1874627658146560:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 13316us (13713us total) trans 4294968021 rc 0/0 00000100:00100000:0.0:1787784778.882862:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1112 00000020:00080000:0.0:1787784778.882868:0:10850:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00080000:0.0:1787784778.882894:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784778.882898:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 219 00000100:00100000:3.0:1787784778.882906:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28756:x1874627658146816:12345-192.168.203.43@tcp:49:cmp.0 00000100:00100000:3.0:1787784778.882999:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28756:x1874627658146816:12345-192.168.203.43@tcp:49:cmp.0 Request processed in 93us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.883006:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 219 00000040:00080000:2.0:1787784778.883496:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784778.883500:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784778.885020:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:2.0:1787784778.885034:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784778.885698:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.885824:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658146944 00000100:00100000:3.0:1787784778.885832:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658146944 00000100:00100000:3.0:1787784778.885887:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 299 00000100:00100000:3.0:1787784778.885897:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:28756:x1874627658146944:12345-192.168.203.43@tcp:101:cmp.0 00010000:00010000:3.0:1787784778.885911:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784778.885930:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10e6 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784778.885954:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x18:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6a8f6c28, ctime: 0x6a8f6e48, blocks: 0xb 00010000:00010000:3.0:1787784778.885985:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=569s v=(0 0) 00010000:00010000:3.0:1787784778.885989:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784778.885999:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.886021:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.886033:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784778.886089:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:28756:x1874627658146944:12345-192.168.203.43@tcp:101:cmp.0 Request processed in 194us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784778.886096:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 299 00000100:00100000:1.0:1787784778.889689:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.889734:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784778.889840:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658147072 00000100:00100000:0.0:1787784778.889849:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658147072 00002000:00100000:0.0:1787784778.889908:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x18:0x0] (0->4095) req@ffff9894035dad00 x1874627658147072/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:659/0 lens 488/0 e 0 to 0 dl 1787784789 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784778.889919:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894035dad00 with x1874627658147072 ext(0->4095) 00002000:00010000:0.0:1787784778.889929:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894035dad00 00000100:00100000:0.0:1787784778.889965:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 100 00000100:00100000:0.0:1787784778.889978:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035dad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658147072:12345-192.168.203.43@tcp:3:cmp.0 00000100:00100000:0.0:1787784778.890228:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035dad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658147072:12345-192.168.203.43@tcp:3:cmp.0 Request processed in 239us (545us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784778.890237:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 100 00002000:00100000:0.0:1787784778.890244:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x18:0x0] (0->4095) req@ffff9894035dad00 x1874627658147072/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:659/0 lens 488/4536 e 0 to 0 dl 1787784789 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784778.890271:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894035dad00 with x1874627658147072 ext(0->4095) 00002000:00010000:0.0:1787784778.890275:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894035dad00 00000100:00100000:3.0:1787784778.891528:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658147328 00000100:00100000:3.0:1787784778.891537:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658147328 00000100:00100000:3.0:1787784778.891643:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 220 00000100:00100000:3.0:1787784778.891654:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:28757:x1874627658147328:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1787784778.892305:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1787784778.892410:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968022 00010000:00010000:3.0:1787784778.892454:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.892470:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.892481:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.892488:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.892501:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.892507:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.892527:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784778.892622:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:28757:x1874627658147328:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 968us (2885us total) trans 4294968023 rc 0/0 00000100:00100000:3.0:1787784778.892641:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 220 00000020:00080000:3.0:1787784778.892647:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1787784778.892711:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784778.892718:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.892725:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784778.892729:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.892737:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.892750:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff989510520000) 00010000:00010000:0.0:1787784778.892753:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787784778.892757:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.892762:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc63e3 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.892777:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784778.892779:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784778.892781:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784778.897723:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.897802:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658147584 00000100:00100000:3.0:1787784778.897813:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658147584 00000100:00100000:3.0:1787784778.897903:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 221 00000100:00100000:3.0:1787784778.897914:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:28756:x1874627658147584:12345-192.168.203.43@tcp:35:cmp.0 00000001:00080000:3.0:1787784778.898227:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968024 00000100:00100000:3.0:1787784778.898325:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:28756:x1874627658147584:12345-192.168.203.43@tcp:35:cmp.0 Request processed in 413us (610us total) trans 4294968025 rc 0/0 00000100:00100000:3.0:1787784778.898334:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 221 00000020:00080000:3.0:1787784778.898339:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1787784778.919180:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784778.919455:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658147712 00000100:00100000:1.0:1787784778.919465:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658147712 00000100:00100000:1.0:1787784778.919521:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1113 00000100:00100000:1.0:1787784778.919531:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:28758:x1874627658147712:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784778.919555:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784778.919576:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc63ea lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10ed expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920002:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920018:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920099:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920117:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920126:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920133:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920588:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920596:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920601:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784778.920650:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00000004:00200000:1.0:1787784778.920655:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784778.920720:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920742:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920751:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920768:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920776:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920782:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784778.920785:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920792:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920804:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff98951c407900) 00010000:00010000:1.0:1787784778.920807:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784778.920813:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920819:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc63f1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920832:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920836:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920840:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920843:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784778.920850:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920854:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920860:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x81b47e8f (ffff989519d44700) 00010000:00010000:1.0:1787784778.920861:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784778.920863:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920867:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc63f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920893:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc63ea lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 97 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920902:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.920908:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:1.0:1787784778.920910:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784778.920913:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:1.0:1787784778.920998:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:28758:x1874627658147712:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1467us (1860us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.921006:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1113 00000100:00100000:1.0:1787784778.927304:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.927364:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658147840 00000100:00100000:2.0:1787784778.927374:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658147840 00000100:00100000:2.0:1787784778.927435:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1114 00000100:00100000:2.0:1787784778.927444:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:28758:x1874627658147840:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784778.927463:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784778.927484:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6406 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10f4 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.927547:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc640d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.927561:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc640d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.927572:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc640d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784778.928229:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 7680 00000004:00200000:2.0:1787784778.928236:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784778.928327:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc640d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.928357:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6406 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10f4 expref: 98 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.928375:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc200/0xb0ce63137acc640d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10f4 expref: 97 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.928383:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff98951c407900) 00010000:00010000:2.0:1787784778.928386:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784778.928390:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc200, rc 0) 00000100:00100000:2.0:1787784778.928453:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:28758:x1874627658147840:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1009us (1157us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784778.928460:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1114 00000100:00100000:2.0:1787784778.935516:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784778.936615:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658148096 00000100:00100000:0.0:1787784778.936691:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658148096 00000100:00100000:0.0:1787784778.936799:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1115 00000100:00100000:0.0:1787784778.936810:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:28758:x1874627658148096:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784778.936830:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784778.936863:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6414 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b10fb expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.936966:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc641b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.936981:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc641b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.936994:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc641b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784778.937006:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00010000:00010000:0.0:1787784778.937189:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6414 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10fb expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.937209:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc641b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10fb expref: 99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.937228:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:0.0:1787784778.937231:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784778.937236:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:0.0:1787784778.937294:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:28758:x1874627658148096:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 486us (1788us total) trans 4294968026 rc 0/0 00000100:00100000:0.0:1787784778.937301:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1115 00000100:00100000:1.0:1787784778.944956:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.945038:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658148224 00000100:00100000:3.0:1787784778.945047:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658148224 00000100:00100000:3.0:1787784778.945106:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:3.0:1787784778.945116:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658148224:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:3.0:1787784778.945198:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.945245:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.945259:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784778.945263:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784778.945268:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.945318:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.945333:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 13 pid: 6669 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.945348:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668404864:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784778.945419:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784778.948009:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668404864:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784778.948037:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.953510:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784778.954290:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658148352 00000100:00100000:2.0:1787784778.954301:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658148352 00010000:00010000:2.0:1787784778.954342:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 14 pid: 6669 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784778.954386:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 334 00000100:00100000:2.0:1787784778.954397:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658148352:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784778.954413:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784778.954435:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 14 pid: 6669 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.954450:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 14 pid: 6669 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.954474:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc63dc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10e6 expref: 14 pid: 6669 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.954492:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784778.954497:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784778.954503:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.954513:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.954549:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784778.954602:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658148352:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 207us (1101us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784778.954610:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 334 00010000:00010000:1.0:1787784778.954971:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955005:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955454:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955465:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955472:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955478:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784778.955482:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955491:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955505:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784778.955512:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc6429 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784778.955593:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658148224:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 10477us (10650us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784778.955606:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:1.0:1787784778.958856:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784778.958956:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658148608 00000100:00100000:0.0:1787784778.958966:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658148608 00000100:00100000:0.0:1787784778.959017:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 222 00000100:00100000:0.0:1787784778.959248:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d4850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:28758:x1874627658148608:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:0.0:1787784778.959328:0:5814:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000401:0x44:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc641b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10fb expref: 99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784778.959577:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968027 00000001:00080000:0.0:1787784778.961388:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968028 00010000:00010000:0.0:1787784778.961462:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.961479:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.961489:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.961497:0:5814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.961534:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.961549:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 102 pid: 5813 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.961563:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668404992:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784778.961666:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.961686:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 102 pid: 5811 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.961708:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403597840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668405120:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784778.961734:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503421840 going to sleep for 11 seconds 00000100:00100000:3.0:1787784778.970576:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784778.970679:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784778.971275:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668404992:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784778.971289:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403597840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668405120:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784778.971317:0:5814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784778.971968:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658148736 00000100:00100000:2.0:1787784778.971976:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658148736 00010000:00010000:2.0:1787784778.972666:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 103 pid: 5811 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784778.973092:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658148864 00000100:00100000:2.0:1787784778.973099:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658148864 00010000:00010000:2.0:1787784778.973119:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 104 pid: 5813 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784778.973175:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:2.0:1787784778.973186:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:2357:x1874627658148736:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784778.973201:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784778.973208:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 104 pid: 5811 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.973220:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 104 pid: 5811 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.973234:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 336 00010000:00010000:2.0:1787784778.973241:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc630a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10a7 expref: 104 pid: 5811 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784778.973244:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:2357:x1874627658148864:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784778.973252:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784778.973257:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 103 pid: 5813 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.973258:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:2.0:1787784778.973263:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.973271:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784778.973277:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784778.973331:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:2357:x1874627658148736:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 147us (2765us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784778.973338:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 335 00010000:00010000:0.0:1787784778.975443:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 102 pid: 5813 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.975472:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc63ff lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10ed expref: 102 pid: 5813 timeout: 670 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.975489:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:0.0:1787784778.975494:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.975503:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.975526:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784778.975561:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784778.975620:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:2357:x1874627658148864:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2377us (4939us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784778.975629:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 336 00010000:00010000:3.0:1787784778.975847:0:5814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.975867:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784778.976398:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968029 00000001:00080000:3.0:1787784778.976627:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968030 00000100:00100000:3.0:1787784778.977271:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668405248:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1787784778.977329:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784778.977363:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033becc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784778.977419:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668405248 00000100:00100000:0.0:1787784778.977425:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668405248 00000100:00100000:0.0:1787784778.977503:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:0.0:1787784778.977512:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5814:x1874627668405248:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1787784778.977590:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5814:x1874627668405248:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 80us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784778.977597:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00000100:00100000:3.0:1787784778.977674:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668405248:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:3.0:1787784778.978052:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968031 00000100:00100000:3.0:1787784778.978110:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400a580 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668405376:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1787784778.978140:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784778.979130:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508d47600 going to sleep for 11 seconds 00000100:00100000:2.0:1787784778.979515:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668405376 00000100:00100000:2.0:1787784778.979523:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668405376 00000100:00100000:2.0:1787784778.979569:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:2.0:1787784778.979579:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5814:x1874627668405376:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1787784778.979810:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5814:x1874627668405376:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 233us (1669us total) trans 4294967365 rc 0/0 00000100:00100000:2.0:1787784778.979821:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00000100:00100000:3.0:1787784778.980839:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400a580 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668405376:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:3.0:1787784778.980914:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968032 00010000:00010000:3.0:1787784778.980962:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.980973:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.980980:0:5814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.980997:0:5814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784778.981002:0:5814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.981025:0:5814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.981049:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:3.0:1787784778.981053:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:3.0:1787784778.981058:0:5814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784778.981064:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6430 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784778.981280:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x45:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787784778.981363:0:5814:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].16, 64 off9312 00000001:00080000:3.0:1787784778.981371:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968033 00010000:00010000:3.0:1787784778.981412:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff989505542e00) 00010000:00010000:3.0:1787784778.981415:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1787784778.981422:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc641b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b10fb expref: 105 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784778.981520:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d4850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:28758:x1874627658148608:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 22276us (22672us total) trans 4294968034 rc 0/0 00000100:00100000:3.0:1787784778.981528:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 222 00000020:00080000:3.0:1787784778.981533:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:1.0:1787784779.250141:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668405504:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784779.250236:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784779.250349:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668405504 00000100:00100000:2.0:1787784779.250357:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668405504 00000100:00100000:2.0:1787784779.250409:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1787784779.250421:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668405504:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784779.250444:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3857711104 avail 3854565376 00000100:00100000:2.0:1787784779.250490:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668405504:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784779.250496:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:1.0:1787784779.250532:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668405632:0@lo:400:kworker.0 00000100:00100000:2.0:1787784779.250546:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668405760:0@lo:400:kworker.0 00000100:00100000:1.0:1787784779.250569:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784779.250586:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784779.250605:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539252d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668406144:0@lo:400:kworker.0 00000100:00100000:1.0:1787784779.250639:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784779.250661:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668405760 00000100:00100000:2.0:1787784779.250667:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668405760 00000100:00100000:1.0:1787784779.250679:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668406016:0@lo:400:kworker.0 00000100:00100000:2.0:1787784779.250699:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406144 00000100:00100000:2.0:1787784779.250703:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406144 00000100:00100000:1.0:1787784779.250707:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784779.250736:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1787784779.250744:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668405760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784779.250749:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668405504:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784779.250769:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784779.250775:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668405888:0@lo:400:kworker.0 00000100:00100000:2.0:1787784779.250809:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668405760:12345-0@lo:400:kworker.0 Request processed in 63us (221us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784779.250809:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784779.250820:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1787784779.250836:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668405888 00000100:00100000:2.0:1787784779.250839:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668405888 00000100:00100000:1.0:1787784779.250846:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1787784779.250855:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668406144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784779.250878:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:2.0:1787784779.250888:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668405888:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784779.250893:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668406144:12345-0@lo:400:kworker.0 Request processed in 39us (263us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784779.250901:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1787784779.250927:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539252d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668406144:0@lo:400:kworker.0 00000100:00100000:2.0:1787784779.250952:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668405888:12345-0@lo:400:kworker.0 Request processed in 64us (142us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784779.250964:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1787784779.250968:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668405888:0@lo:400:kworker.0 00000100:00100000:2.0:1787784779.251006:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668405760:0@lo:400:kworker.0 00000100:00100000:3.0:1787784779.251047:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668405632 00000100:00100000:3.0:1787784779.251052:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668405632 00000100:00100000:3.0:1787784779.251095:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 558 00000100:00100000:3.0:1787784779.251102:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668405632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784779.251173:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668405632:12345-0@lo:400:kworker.0 Request processed in 72us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784779.251187:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 558 00000100:00100000:2.0:1787784779.251431:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668405632:0@lo:400:kworker.0 00000100:00080000:0.0:1787784779.251469:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784779.251486:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406016 00000100:00100000:0.0:1787784779.251491:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406016 00000100:00080000:2.0:1787784779.251524:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784779.251539:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:0.0:1787784779.251547:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668406016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784779.251584:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668406016:12345-0@lo:400:kworker.0 Request processed in 38us (879us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784779.251589:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1787784779.257333:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668406016:0@lo:400:kworker.0 00010000:00010000:2.0:1787784779.698512:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784780.726907:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784781.748225:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784782.321796:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668406272:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784782.321869:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784782.322217:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406272 00000100:00100000:2.0:1787784782.322226:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406272 00000100:00100000:2.0:1787784782.322275:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1787784782.322283:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403597480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668406272:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784782.322322:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784782.322373:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403597480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668406272:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784782.322379:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1787784782.322425:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668406272:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784782.322483:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784782.602665:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784782.602712:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784782.602732:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658148992 00000100:00100000:1.0:1787784782.602754:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658148992 00000100:00100000:2.0:1787784782.602804:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784782.602816:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 559 00000100:00100000:1.0:1787784782.602827:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658148992:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784782.602841:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784782.603085:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658148992:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 259us (428us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784782.603093:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 559 00000100:00100000:2.0:1787784782.610227:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784782.610267:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784782.610286:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658149504 00000100:00100000:1.0:1787784782.610297:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658149504 00000100:00100000:1.0:1787784782.610418:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658149120 00000100:00100000:1.0:1787784782.610422:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658149120 00000100:00100000:1.0:1787784782.610446:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:1.0:1787784782.610454:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2356:x1874627658149504:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784782.610528:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2356:x1874627658149504:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 75us (7726us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784782.610536:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:1.0:1787784782.610657:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 387 00000100:00100000:1.0:1787784782.610666:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:2356:x1874627658149120:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784782.610696:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:2356:x1874627658149120:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784782.610703:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 387 00000100:00080000:1.0:1787784782.611013:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784782.613383:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658149248 00000100:00100000:0.0:1787784782.613540:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658149248 00000100:00100000:2.0:1787784782.613727:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784782.613769:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658149760 00000100:00100000:2.0:1787784782.613773:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658149760 00000100:00100000:2.0:1787784782.613835:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658149376 00000100:00100000:2.0:1787784782.613839:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658149376 00000100:00100000:2.0:1787784782.613856:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658149632 00000100:00100000:2.0:1787784782.613859:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658149632 00000100:00100000:2.0:1787784782.613875:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:2.0:1787784782.613884:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658149760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784782.613951:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658149760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 68us (11111us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784782.615502:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:0.0:1787784782.622757:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:0.0:1787784782.622785:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658149376:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784782.622874:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658149376:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 104us (12607us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784782.622883:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:0.0:1787784782.622913:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 342 00000100:00100000:0.0:1787784782.622921:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658149632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784782.622948:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658149632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (9225us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784782.622955:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 342 00000100:00080000:2.0:1787784782.624562:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784782.624570:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784782.629095:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 343 00000100:00100000:1.0:1787784782.629115:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658149248:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784782.630624:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658149248:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1511us (27908us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784782.630635:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 343 00010000:00010000:2.0:1787784782.769952:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784782.961506:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784782.961520:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784782.961773:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 31 count 2 00000040:00080000:2.0:1787784782.961781:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784783.794359:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784783.985548:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967363 is committed 00000001:00080000:0.0:1787784783.985571:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967365 is committed 00000001:00080000:2.0:1787784784.001417:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968029 is committed 00000001:00080000:2.0:1787784784.001445:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968021 is committed 00000020:00080000:2.0:1787784784.001451:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9895407276c0 refcount 2 committed 0 result 0 batchid 21 00000020:00080000:2.0:1787784784.001457:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989508cfc700 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989540727840 00000020:00080000:2.0:1787784784.001461:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989508cfc300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989510ac7800 00000001:00080000:2.0:1787784784.001495:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968034 is committed 00000020:00080000:2.0:1787784784.001505:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9895033be900 refcount 2 committed 0 result 0 batchid 22 00000020:00080000:2.0:1787784784.001508:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989518ed7f00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9895217020c0 00000020:00080000:2.0:1787784784.001511:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989518ed7700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895401a2000 00000001:00080000:2.0:1787784784.001527:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968023 is committed 00000020:00080000:2.0:1787784784.001539:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 22 committed batchid 20 00000020:00080000:2.0:1787784784.002930:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 20 00000100:00100000:2.0:1787784784.369444:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668406400:0@lo:400:kworker.0 00000100:00100000:3.0:1787784784.369494:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539252d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668406656:0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.369528:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784784.369579:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784784.369617:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668406912:0@lo:400:kworker.0 00000100:00100000:3.0:1787784784.369647:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784784.369658:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668406784:0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.369659:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784784.369707:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784784.369721:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668406528:0@lo:400:kworker.0 00000100:00100000:3.0:1787784784.369728:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784784.369770:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784784.369873:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406400 00000100:00100000:1.0:1787784784.369882:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406400 00000100:00100000:1.0:1787784784.369928:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 560 00000100:00100000:1.0:1787784784.369938:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668406400:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784784.370079:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668406400:12345-0@lo:400:kworker.0 Request processed in 141us (551us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784784.370087:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 560 00000100:00100000:3.0:1787784784.370470:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406656 00000100:00100000:3.0:1787784784.370479:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406656 00000100:00100000:3.0:1787784784.370549:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406912 00000100:00100000:3.0:1787784784.370554:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406912 00000100:00100000:3.0:1787784784.370570:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406528 00000100:00100000:3.0:1787784784.370573:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406528 00000100:00100000:3.0:1787784784.370605:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1787784784.370615:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668406656:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1787784784.370709:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00080000:1.0:1787784784.372252:0:8078:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 575459103226ns, next at 576459100886ns, rc = 0 00000100:00100000:1.0:1787784784.372294:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:1.0:1787784784.373229:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668406912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.377076:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668407040:0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.377168:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784784.377210:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668406784 00000100:00100000:2.0:1787784784.377218:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668406784 00000100:00100000:2.0:1787784784.377274:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668407040 00000100:00100000:2.0:1787784784.377277:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668407040 00000100:00100000:2.0:1787784784.377303:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1787784784.377312:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668406784:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.377358:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668406784:12345-0@lo:400:kworker.0 Request processed in 47us (7639us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784784.377366:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1787784784.377380:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1787784784.377386:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668407040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.377414:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668407040:12345-0@lo:400:kworker.0 Request processed in 29us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784784.377420:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1787784784.377455:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967365 gen 2 00000004:00080000:2.0:1787784784.377461:0:3276:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967364 committed transno 4294967365 00000004:00080000:2.0:1787784784.377479:0:3276:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967365 committed transno 4294967365 00000100:00100000:2.0:1787784784.377495:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668407040:0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.377511:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668406400:0@lo:400:kworker.0 00000100:00100000:2.0:1787784784.382332:0:5811:0:(service.c:3462:ptlrpc_start_thread()) mdt[0] started 4 min 3 max 116 00000100:00100000:2.0:1787784784.382348:0:5811:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt00_004' 00000100:00100000:0.0:1787784784.384273:0:5813:0:(service.c:3462:ptlrpc_start_thread()) mdt[0] started 4 min 3 max 116 00000100:00100000:0.0:1787784784.384297:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:0.0:1787784784.384311:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668406528:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784784.384405:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668406528:12345-0@lo:400:kworker.0 Request processed in 98us (14637us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784784.384413:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00080000:0.0:1787784784.384440:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784784.384445:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784784.384486:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668406528:0@lo:400:kworker.0 00000100:00100000:3.0:1787784784.387472:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784784.387510:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668407168:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784784.387613:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784784.387687:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668407168 00000100:00100000:3.0:1787784784.387697:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668407168 00000100:00100000:3.0:1787784784.387757:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:3.0:1787784784.387768:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668407168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784784.387822:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784784.387900:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668407168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 132us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784784.387908:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:3.0:1787784784.387989:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668407168:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784784.388050:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668406784:0@lo:400:kworker.0 00000100:00100000:3.0:1787784784.389001:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668406656:12345-0@lo:400:kworker.0 Request processed in 18380us (19418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784784.389014:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:2.0:1787784784.390081:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539252d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668406656:0@lo:400:kworker.0 00000100:00100000:0.0:1787784784.393257:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668406912:0@lo:400:kworker.0 00000100:00100000:1.0:1787784784.393365:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668406912:12345-0@lo:400:kworker.0 Request processed in 20142us (23706us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784784.393377:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00010000:00010000:1.0:1787784784.818525:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784784.872629:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784784.879215:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658149888 00000100:00100000:0.0:1787784784.879231:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658149888 00000100:00100000:0.0:1787784784.879350:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1116 00000100:00100000:0.0:1787784784.879361:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:28706:x1874627658149888:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1787784784.879470:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:28706:x1874627658149888:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 111us (6850us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784784.879479:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1116 00000100:00080000:1.0:1787784784.879669:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784785.841297:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784786.865282:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787784787.213007:0:20487:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 18:53:04 (1787784784) 00000100:00100000:0.0:1787784787.441404:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668407296:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784787.441614:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784787.441748:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668407296 00000100:00100000:2.0:1787784787.441756:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668407296 00000100:00100000:2.0:1787784787.441815:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1787784787.441822:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668407296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784787.441853:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784787.441888:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668407296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.441892:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00080000:1.0:1787784787.442051:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784787.442102:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668407296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784787.724633:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.724667:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.724689:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.724706:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.724725:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.724743:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784787.724770:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150016 00000100:00100000:2.0:1787784787.724778:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784787.724782:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150016 00000100:00100000:2.0:1787784787.724813:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150272 00000100:00100000:2.0:1787784787.724819:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150272 00000100:00100000:1.0:1787784787.724844:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 561 00000100:00100000:1.0:1787784787.724857:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658150016:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784787.724863:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150784 00000100:00100000:2.0:1787784787.724867:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150784 00000100:00100000:2.0:1787784787.724881:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150400 00000100:00100000:2.0:1787784787.724883:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150400 00000100:00100000:2.0:1787784787.724911:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150656 00000100:00100000:2.0:1787784787.724914:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150656 00000100:00100000:1.0:1787784787.724925:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658150016:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 71us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.724932:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 346 00000100:00100000:1.0:1787784787.724937:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 561 00000100:00100000:2.0:1787784787.724943:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2354:x1874627658150272:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784787.724975:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:2.0:1787784787.724980:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2354:x1874627658150272:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784787.724987:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658150784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784787.725007:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 346 00000100:00100000:1.0:1787784787.725019:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658150784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784787.725027:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:2.0:1787784787.725034:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 348 00000100:00100000:2.0:1787784787.725045:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658150400:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784787.725050:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:1.0:1787784787.725060:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658150656:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784787.725077:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658150400:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 35us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.725085:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 348 00000100:00100000:1.0:1787784787.725093:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658150656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (314us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784787.725101:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:2.0:1787784787.725168:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150528 00000100:00100000:2.0:1787784787.725173:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150528 00000100:00100000:2.0:1787784787.725203:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150144 00000100:00100000:2.0:1787784787.725207:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150144 00000100:00100000:2.0:1787784787.725233:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:2.0:1787784787.725241:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2354:x1874627658150528:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784787.725290:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2354:x1874627658150528:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 50us (599us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.725297:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:2.0:1787784787.725312:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 392 00000100:00100000:2.0:1787784787.725320:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:2354:x1874627658150144:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:1.0:1787784787.725330:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784787.725335:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784787.725338:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784787.725812:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:2354:x1874627658150144:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 493us (1085us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.725823:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 392 00000100:00100000:2.0:1787784787.747779:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784787.747878:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658150912 00000100:00100000:3.0:1787784787.747887:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658150912 00000100:00100000:3.0:1787784787.747991:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1117 00000100:00100000:3.0:1787784787.748001:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:29372:x1874627658150912:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1787784787.748106:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:29372:x1874627658150912:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 105us (336us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784787.748114:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1117 00000100:00100000:1.0:1787784787.762600:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.763381:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658151040 00000100:00100000:0.0:1787784787.763392:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658151040 00000100:00100000:0.0:1787784787.763454:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1118 00000100:00100000:0.0:1787784787.763463:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:29373:x1874627658151040:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784787.763546:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.763626:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6437 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1102 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.763746:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.763761:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.763774:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.763829:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.763856:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.763864:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764406:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764415:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764422:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764428:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a3200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.764433:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764439:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764451:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784787.764454:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.764458:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764464:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc643e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764476:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764500:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764505:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764510:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.764512:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764518:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764524:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff989503a6d200) 00010000:00010000:0.0:1787784787.764527:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.764529:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764535:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6445 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764722:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6437 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1102 expref: 94 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764740:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784787.764743:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.764746:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6437 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1102 expref: 94 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.764756:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:0.0:1787784787.764830:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:29373:x1874627658151040:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1368us (2238us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784787.764837:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1118 00000100:00100000:2.0:1787784787.774667:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.775289:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658151296 00000100:00100000:0.0:1787784787.775298:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658151296 00000100:00100000:0.0:1787784787.775390:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1119 00000100:00100000:0.0:1787784787.775400:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:29373:x1874627658151296:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784787.775454:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784787.775478:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6246 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b106f expref: 93 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.775496:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784787.775500:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784787.775810:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.775868:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.775874:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.775950:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 42 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.775960:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 42 pid: 5813 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784787.775971:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668407424:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784787.776056:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784787.779664:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668407424:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784787.779697:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784787.779806:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784787.780809:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658151424 00000100:00100000:3.0:1787784787.780872:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658151424 00010000:00010000:3.0:1787784787.780974:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 43 pid: 5813 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784787.781050:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:3.0:1787784787.781059:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2357:x1874627658151424:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784787.781074:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784787.781080:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 43 pid: 5813 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.781092:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 43 pid: 5813 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.781128:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6270 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b107d expref: 43 pid: 5813 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.781143:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:3.0:1787784787.781163:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.781180:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.781191:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.781212:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784787.781269:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+42:2357:x1874627658151424:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 211us (1464us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784787.781278:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 337 00010000:00010000:0.0:1787784787.781338:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.781347:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.781372:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.781382:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.781391:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784787.782711:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 512 00010000:00010000:0.0:1787784787.782722:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.782728:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.782733:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.782736:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.782739:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.782750:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.782758:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784787.782760:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.782763:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.782766:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc644c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784787.782780:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d9ee262e reply state 0000000048c7812b transno 4294968035 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784787.782789:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784787.782883:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:29373:x1874627658151296:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 7482us (8223us total) trans 4294968035 rc 0/0 00000100:00100000:0.0:1787784787.782891:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1119 00010000:00010000:0.0:1787784787.784962:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.784978:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.784986:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.784993:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.784997:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.785022:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.785037:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff989503a6d200) 00010000:00010000:0.0:1787784787.785040:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.785044:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.785050:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6453 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784787.791240:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784787.791327:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658151552 00000100:00100000:3.0:1787784787.791337:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658151552 00000100:00100000:3.0:1787784787.791405:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1120 00000100:00100000:3.0:1787784787.791416:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+93:29374:x1874627658151552:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784787.791492:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784787.791517:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc645a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1109 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.792242:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6461 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.792277:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6461 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.792289:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6461 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784787.792358:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784787.792362:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 29 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784787.792386:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6461 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.792411:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc645a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1109 expref: 95 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.792432:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6461 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1109 expref: 94 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.792441:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:3.0:1787784787.792443:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784787.792447:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784787.792515:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:29374:x1874627658151552:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1101us (1284us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784787.792522:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1120 00000100:00100000:2.0:1787784787.795063:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.795179:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658151680 00000100:00100000:0.0:1787784787.795188:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658151680 00000100:00100000:0.0:1787784787.795232:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1121 00000100:00100000:0.0:1787784787.795241:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+94:29374:x1874627658151680:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784787.795259:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.795275:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6468 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1110 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795327:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795338:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795348:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795360:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795377:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795383:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795474:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2c00/0xb0ce63137acc647d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795480:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2c00/0xb0ce63137acc647d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795486:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2c00/0xb0ce63137acc647d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784787.795509:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 512 00000004:00200000:0.0:1787784787.795511:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784787.795528:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2c00/0xb0ce63137acc647d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795539:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795551:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795557:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795562:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.795566:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795589:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795600:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:0.0:1787784787.795603:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.795607:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795613:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc646f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795625:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795631:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795636:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795641:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a3200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.795644:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795650:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795657:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff989503a6d200) 00010000:00010000:0.0:1787784787.795659:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.795662:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795676:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6476 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795703:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc6468 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1110 expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795714:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2c00/0xb0ce63137acc647d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1110 expref: 95 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.795721:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.795723:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784787.795725:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2c00, rc 0) 00000100:00100000:0.0:1787784787.795774:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29374:x1874627658151680:12345-192.168.203.43@tcp:101:bash.0 Request processed in 534us (719us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784787.795780:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1121 00000100:00100000:2.0:1787784787.799671:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.799740:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658151808 00000100:00100000:0.0:1787784787.799749:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658151808 00000100:00100000:0.0:1787784787.799807:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1122 00000100:00100000:0.0:1787784787.799818:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29374:x1874627658151808:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784787.799836:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.799856:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6484 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1117 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.799952:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.799967:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.799985:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800001:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800010:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800016:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800083:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800090:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800097:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800104:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.800108:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800115:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800127:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.800129:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.800134:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800190:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc648b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800205:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800211:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800215:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800220:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.800222:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800228:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800234:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff989505542e00) 00010000:00010000:0.0:1787784787.800237:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.800240:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800245:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6492 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800288:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6484 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1117 expref: 96 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800845:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.800849:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.800854:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6484 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1117 expref: 96 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.800867:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2200, rc 0) 00000100:00100000:0.0:1787784787.800927:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29374:x1874627658151808:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1110us (1264us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784787.800936:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1122 00000100:00100000:0.0:1787784787.810544:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784787.811105:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658151936 00000100:00100000:3.0:1787784787.811116:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658151936 00000100:00100000:2.0:1787784787.811381:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1123 00000100:00100000:2.0:1787784787.811393:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+42:29375:x1874627658151936:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784787.811412:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784787.811428:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6499 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b111e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.811477:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.811498:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.811507:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784787.811558:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784787.811562:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 29 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784787.811692:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.811716:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6499 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 44 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.811732:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.811738:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:2.0:1787784787.811740:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784787.811742:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784787.811795:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+43:29375:x1874627658151936:12345-192.168.203.43@tcp:101:rm.0 Request processed in 406us (1263us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.811800:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1123 00000100:00100000:2.0:1787784787.816276:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.816371:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658152064 00000100:00100000:0.0:1787784787.816381:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658152064 00000100:00100000:0.0:1787784787.816438:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1124 00000100:00100000:0.0:1787784787.816449:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:29375:x1874627658152064:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784787.816467:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.816557:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc64a7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1125 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.816864:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.816881:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.816893:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.816926:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.816934:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.816942:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817048:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc64bc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817058:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc64bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817066:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc64bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784787.817096:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 512 00000004:00200000:0.0:1787784787.817100:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784787.817122:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc64bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817134:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817140:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817188:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817205:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.817209:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817217:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817228:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:0.0:1787784787.817231:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.817235:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817242:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc64ae lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817266:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817272:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817279:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817284:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.817287:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817294:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817313:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff989505542e00) 00010000:00010000:0.0:1787784787.817316:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.817319:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817325:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc64b5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817354:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc64a7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1125 expref: 45 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817367:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3a00/0xb0ce63137acc64bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1125 expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.817375:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.817377:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784787.817380:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a3a00, rc 0) 00000100:00100000:0.0:1787784787.817431:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29375:x1874627658152064:12345-192.168.203.43@tcp:101:rm.0 Request processed in 984us (1162us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784787.817439:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1124 00000100:00100000:1.0:1787784787.821141:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.821467:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658152192 00000100:00100000:0.0:1787784787.821475:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658152192 00000100:00100000:0.0:1787784787.821520:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1125 00000100:00100000:0.0:1787784787.821527:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29375:x1874627658152192:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784787.821544:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.821566:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc64c3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b112c expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821655:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821665:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821673:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821682:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821686:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821690:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821734:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821738:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821741:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821746:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.821749:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821760:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821768:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.821770:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.821774:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821777:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64ca lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821788:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821792:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821796:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821798:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.821800:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821803:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821806:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff989505542f00) 00010000:00010000:0.0:1787784787.821808:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.821809:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821816:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64d1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821838:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc64c3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b112c expref: 45 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821845:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.821846:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.821848:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc64c3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b112c expref: 45 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.821855:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a3200, rc 0) 00000100:00100000:0.0:1787784787.821903:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29375:x1874627658152192:12345-192.168.203.43@tcp:101:rm.0 Request processed in 376us (772us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784787.821908:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1125 00000100:00100000:1.0:1787784787.824427:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.824494:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658152320 00000100:00100000:2.0:1787784787.824503:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658152320 00000100:00100000:2.0:1787784787.824558:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1126 00000100:00100000:2.0:1787784787.824580:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29375:x1874627658152320:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784787.824598:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784787.824617:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc64d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1133 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824677:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824690:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824701:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824718:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824726:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824733:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824794:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824802:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824808:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824814:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784787.824818:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824825:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824837:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:2.0:1787784787.824840:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784787.824845:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824851:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc64df lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824863:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824869:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824875:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824880:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784787.824882:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824888:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824894:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff989530cb1200) 00010000:00010000:2.0:1787784787.824896:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784787.824899:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824904:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc64e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824931:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc64d8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1133 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824940:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:2.0:1787784787.824942:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784787.824946:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc64d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1133 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.824954:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784787.825007:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29375:x1874627658152320:12345-192.168.203.43@tcp:101:rm.0 Request processed in 440us (586us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784787.825015:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1126 00000100:00100000:2.0:1787784787.840755:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.840811:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658152448 00000100:00100000:0.0:1787784787.840820:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658152448 00000100:00100000:0.0:1787784787.840877:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1127 00000100:00100000:0.0:1787784787.840887:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29376:x1874627658152448:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784787.840906:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.840925:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc64ed lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b113a expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.840990:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841003:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841029:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841045:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841115:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841125:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841230:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841239:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841246:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841252:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.841257:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841264:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841276:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.841279:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.841283:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841296:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc64f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841313:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841319:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841324:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841329:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.841333:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841338:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841346:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff989505542f00) 00010000:00010000:0.0:1787784787.841348:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.841351:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841356:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc64fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841386:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc64ed lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b113a expref: 45 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841408:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.841411:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.841415:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc64ed lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b113a expref: 45 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.841426:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a3200, rc 0) 00000100:00100000:0.0:1787784787.841486:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29376:x1874627658152448:12345-192.168.203.43@tcp:101:cp.0 Request processed in 600us (739us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784787.841493:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1127 00000100:00100000:1.0:1787784787.844433:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.844698:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658152576 00000100:00100000:0.0:1787784787.844707:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658152576 00000100:00100000:0.0:1787784787.844766:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1128 00000100:00100000:0.0:1787784787.844776:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29376:x1874627658152576:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784787.844794:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.844814:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6502 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1141 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.844900:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.844913:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.844923:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.844938:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.844951:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.844958:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845012:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845019:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845025:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845031:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.845034:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845042:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845053:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.845056:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.845061:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845068:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6509 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845082:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845094:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845100:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845105:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.845119:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845125:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845133:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff989505542e00) 00010000:00010000:0.0:1787784787.845136:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.845138:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845143:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6510 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845210:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6502 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1141 expref: 45 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845219:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.845222:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.845225:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6502 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1141 expref: 45 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.845247:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2200, rc 0) 00000100:00100000:0.0:1787784787.845305:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29376:x1874627658152576:12345-192.168.203.43@tcp:101:cp.0 Request processed in 530us (879us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784787.845312:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1128 00000100:00100000:1.0:1787784787.849953:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784787.852349:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658152832 00000100:00100000:3.0:1787784787.852363:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658152832 00000100:00100000:3.0:1787784787.852437:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1129 00000100:00100000:3.0:1787784787.852448:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29376:x1874627658152832:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784787.852481:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784787.852502:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6517 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1148 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.852649:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.852664:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.852676:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.852691:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.852699:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.852706:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784787.853762:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x16:0x0], index 0: 0 00010000:00010000:3.0:1787784787.854352:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854367:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854376:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854424:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854432:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854439:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854445:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784787.854449:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854457:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854469:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:3.0:1787784787.854473:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784787.854478:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854484:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc651e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784787.854497:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f6639f75 reply state 00000000812a1c71 transno 4294968036 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784787.854505:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787784787.854537:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6517 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854941:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784787.854951:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:3.0:1787784787.854954:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784787.854957:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784787.855084:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29376:x1874627658152832:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2637us (5138us total) trans 4294968036 rc 0/0 00000100:00100000:3.0:1787784787.855092:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1129 00000100:00100000:1.0:1787784787.861091:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.861530:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658152960 00000100:00100000:2.0:1787784787.861540:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658152960 00000100:00100000:2.0:1787784787.861618:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 301 00000100:00100000:2.0:1787784787.861627:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658152960:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784787.861644:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784787.861664:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b114f expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784787.861702:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784787.861722:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=578s v=(0 0) 00010000:00010000:2.0:1787784787.861726:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784787.861741:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.861775:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 9 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.861788:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784787.861853:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658152960:12345-192.168.203.43@tcp:101:cp.0 Request processed in 228us (779us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.861861:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 301 00010000:00010000:0.0:1787784787.863401:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.863416:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.863430:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.863437:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.863442:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.863450:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.863471:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff9895198b5900) 00010000:00010000:0.0:1787784787.863474:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.863479:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.863493:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6525 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784787.866740:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.867445:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658153088 00000100:00100000:2.0:1787784787.867456:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658153088 00000100:00100000:2.0:1787784787.867511:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 302 00000100:00100000:2.0:1787784787.867521:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:29376:x1874627658153088:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784787.867538:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784787.867554:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1156 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.867599:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.867610:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784787.867614:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784787.867619:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.867654:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.867668:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 10 pid: 6669 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784787.867682:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668407552:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784787.867776:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784787.872423:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.872512:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658153216 00000100:00100000:0.0:1787784787.872533:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658153216 00010000:00010000:0.0:1787784787.872583:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 11 pid: 6669 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784787.872628:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 338 00000100:00100000:0.0:1787784787.872638:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658153216:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784787.872653:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784787.872679:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 11 pid: 6669 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.872692:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 11 pid: 6669 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.872717:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784787.872721:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784787.872729:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.872740:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.872779:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784787.872793:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668407680:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784787.872870:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ebb40 going to sleep for 11 seconds 00010000:00010000:2.0:1787784787.873355:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc653a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b114f expref: 11 pid: 6669 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784787.873372:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668407552:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784787.873395:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.873405:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784787.873446:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:29376:x1874627658153088:12345-192.168.203.43@tcp:101:cp.0 Request processed in 5927us (6714us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.873454:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 302 00000100:00100000:0.0:1787784787.884365:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668407680:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784787.884395:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784787.884448:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658153216:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 11811us (12034us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784787.884460:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 338 00010000:00010000:1.0:1787784787.889762:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784787.901519:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.901629:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658153472 00000100:00100000:0.0:1787784787.901639:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658153472 00000100:00100000:0.0:1787784787.901694:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 223 00000100:00100000:0.0:1787784787.901705:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29376:x1874627658153472:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:0.0:1787784787.902041:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29376:x1874627658153472:12345-192.168.203.43@tcp:35:cp.0 Request processed in 337us (530us total) trans 4294968037 rc 0/0 00000100:00100000:0.0:1787784787.902049:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 223 00000100:00100000:1.0:1787784787.981480:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784787.981583:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658153600 00000100:00100000:0.0:1787784787.981593:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658153600 00000100:00100000:0.0:1787784787.981656:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1130 00000100:00100000:0.0:1787784787.981666:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29379:x1874627658153600:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784787.981697:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784787.981720:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6548 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b115d expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981786:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981800:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981813:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981847:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981855:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981862:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981954:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981963:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.981969:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784787.981999:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:0.0:1787784787.982004:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784787.982292:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982304:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982312:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982319:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982336:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982342:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.982346:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982354:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982366:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784787.982369:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.982373:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982378:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc654f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982392:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982399:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982404:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982409:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784787.982412:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982429:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982438:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff989505542e00) 00010000:00010000:0.0:1787784787.982440:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784787.982443:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982448:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6556 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982481:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6548 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 97 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982495:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784787.982501:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:0.0:1787784787.982503:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784787.982506:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a3200, rc 0) 00000100:00100000:0.0:1787784787.982784:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:29379:x1874627658153600:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1118us (1311us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784787.982791:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1130 00000100:00100000:1.0:1787784787.986276:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.986427:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658153728 00000100:00100000:2.0:1787784787.986436:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658153728 00000100:00100000:2.0:1787784787.986486:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 303 00000100:00100000:2.0:1787784787.986496:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658153728:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784787.986511:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784787.986527:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc6564 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1164 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784787.986594:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668407808:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784787.986674:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f0c0 going to sleep for 11 seconds 00002000:00010000:2.0:1787784787.988712:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784787.988718:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb mtime: 0 -> 1787784787 00002000:00010000:2.0:1787784787.988721:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb atime: 0 -> 1787784787 00002000:00010000:2.0:1787784787.988723:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb ctime: 0 -> 1787784787 00000100:00100000:2.0:1787784787.988741:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bcf00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668407808:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784787.988771:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc6564 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1164 expref: 16 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.988793:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057ff88f9/0xb0ce63137acc6564 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1164 expref: 16 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.988805:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784787.988856:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658153728:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2361us (2588us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784787.988865:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 303 00000100:00100000:1.0:1787784787.996427:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784787.997096:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658153856 00000100:00100000:2.0:1787784787.997105:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658153856 00000100:00100000:2.0:1787784787.997189:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1131 00000100:00100000:2.0:1787784787.997198:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:29379:x1874627658153856:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784787.997216:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784787.997235:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc656b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b116b expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.997306:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.997321:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.997333:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784787.997386:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 512 00000004:00200000:2.0:1787784787.997391:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784787.997409:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.997434:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc656b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.997451:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 97 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784787.997459:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:2.0:1787784787.997462:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784787.997465:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:2.0:1787784787.997524:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:29379:x1874627658153856:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 327us (1106us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784787.997531:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1131 00000100:00100000:1.0:1787784788.010744:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.010826:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658153984 00000100:00100000:3.0:1787784788.010835:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658153984 00000100:00100000:3.0:1787784788.010888:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:3.0:1787784788.010898:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658153984:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784788.010915:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784788.010932:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc6579 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1172 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784788.010985:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adf840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668407936:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784788.011079:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5840 going to sleep for 11 seconds 00000100:00100000:2.0:1787784788.015093:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668407936:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784788.015133:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc6579 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1172 expref: 16 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.015436:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc6579 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1172 expref: 16 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.015452:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:2.0:1787784788.015550:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658153984:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4653us (4815us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784788.015560:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:2.0:1787784788.028026:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.028446:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658154240 00000100:00100000:3.0:1787784788.028456:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658154240 00000100:00100000:3.0:1787784788.028624:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1132 00000100:00100000:3.0:1787784788.028635:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:29379:x1874627658154240:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784788.028879:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].31, 136 off12408 00000100:00100000:3.0:1787784788.028956:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:29379:x1874627658154240:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 323us (938us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784788.028964:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1132 00000040:00080000:2.0:1787784788.081396:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784788.081405:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784788.081461:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 32 count 3 00000040:00080000:2.0:1787784788.081467:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784788.081774:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400a580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668408064:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784788.083044:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f300 going to sleep for 11 seconds 00000100:00100000:2.0:1787784788.093397:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400a580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668408064:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784788.100620:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.104062:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658154368 00000100:00100000:0.0:1787784788.104079:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658154368 00000100:00100000:0.0:1787784788.104184:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1133 00000100:00100000:0.0:1787784788.104195:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29388:x1874627658154368:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784788.104248:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784788.104360:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29388:x1874627658154368:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 167us (3748us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784788.104368:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1133 00000100:00100000:3.0:1787784788.111653:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.111860:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658154624 00000100:00100000:0.0:1787784788.111869:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658154624 00000100:00100000:0.0:1787784788.111932:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1134 00000100:00100000:0.0:1787784788.111942:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29388:x1874627658154624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784788.111962:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784788.111983:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6580 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1179 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.112137:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6587 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.112561:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6587 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.112585:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6587 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784788.112599:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1787784788.112731:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6580 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1179 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.113305:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6587 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1179 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.113332:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:0.0:1787784788.113335:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784788.113339:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2400, rc 0) 00000100:00100000:0.0:1787784788.113450:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29388:x1874627658154624:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1509us (1806us total) trans 4294968038 rc 0/0 00000100:00100000:0.0:1787784788.113459:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1134 00000100:00100000:1.0:1787784788.116943:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.118630:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.118734:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658154880 00000100:00100000:0.0:1787784788.118749:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658154880 00000100:00100000:0.0:1787784788.118805:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:0.0:1787784788.118814:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658154880:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784788.118853:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.118871:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.118880:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x16:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784788.118884:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784788.118889:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.118919:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.118933:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 9 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784788.118959:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668408192:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784788.119043:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ebc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784788.122995:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668408192:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784788.123033:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784788.123402:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658154752 00000100:00100000:2.0:1787784788.123409:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658154752 00000100:00100000:2.0:1787784788.123468:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 48 00000100:00100000:2.0:1787784788.123477:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2354:x1874627658154752:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784788.123492:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784788.123513:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6587 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1179 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.123528:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:2.0:1787784788.123533:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784788.123596:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2354:x1874627658154752:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 120us (6662us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784788.123604:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 48 00000100:00100000:3.0:1787784788.125840:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784788.126027:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658155008 00000100:00100000:1.0:1787784788.126035:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658155008 00002000:00100000:1.0:1787784788.126080:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x16:0x0] (0->157) req@ffff98951b45e1c0 x1874627658155008/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:669/0 lens 648/0 e 0 to 0 dl 1787784799 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784788.126091:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b45e1c0 with x1874627658155008 ext(0->157) 00010000:00010000:1.0:1787784788.126100:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 10 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784788.126116:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 10 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784788.126125:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff98951b45e1c0 00000100:00100000:1.0:1787784788.126201:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 12 00000100:00100000:1.0:1787784788.126210:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658155008:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784788.126697:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658155008:12345-192.168.203.43@tcp:4:cp.0 Request processed in 487us (862us total) trans 4294967354 rc 0/0 00000100:00100000:1.0:1787784788.126715:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 12 00002000:00100000:1.0:1787784788.126729:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x16:0x0] (0->157) req@ffff98951b45e1c0 x1874627658155008/t4294967354(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:669/0 lens 648/448 e 0 to 0 dl 1787784799 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784788.126745:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b45e1c0 with x1874627658155008 ext(0->157) 00010000:00010000:1.0:1787784788.126751:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 12 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784788.126763:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 12 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784788.126769:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff98951b45e1c0 00000100:00100000:0.0:1787784788.136121:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.136514:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658155136 00000100:00100000:3.0:1787784788.136524:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658155136 00010000:00010000:3.0:1787784788.136577:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 12 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784788.136620:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 339 00000100:00100000:3.0:1787784788.136630:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658155136:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784788.136644:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784788.136666:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 12 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.136679:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 12 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.136702:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc6541 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1156 expref: 12 pid: 13825 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.136717:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784788.136721:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784788.136727:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.136765:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.136799:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784788.136862:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658155136:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 234us (761us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784788.136877:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 339 00010000:00010000:2.0:1787784788.137555:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137583:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137614:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137622:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137630:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137637:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784788.137641:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137649:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137661:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.137668:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d8922d23/0xb0ce63137acc6595 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784788.137731:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658154880:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 18917us (19105us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784788.137750:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:0.0:1787784788.144423:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.144521:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658155392 00000100:00100000:3.0:1787784788.144530:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658155392 00000100:00100000:3.0:1787784788.144747:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1135 00000100:00100000:3.0:1787784788.144759:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29388:x1874627658155392:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784788.144894:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29388:x1874627658155392:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 137us (479us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784788.144902:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1135 00000100:00100000:1.0:1787784788.155973:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784788.156611:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658155520 00000100:00100000:2.0:1787784788.156621:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658155520 00000100:00100000:2.0:1787784788.156679:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00100000:2.0:1787784788.156688:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658155520:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784788.156705:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784788.156724:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1180 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784788.156746:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 158, mtime: 0x6a8f6e53, atime: 0x0, ctime: 0x6a8f6e53, blocks: 0x1 00010000:00010000:2.0:1787784788.156766:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=579s v=(0 0) 00010000:00010000:2.0:1787784788.156771:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784788.156781:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.156797:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.156811:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784788.156872:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658155520:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 185us (908us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784788.156879:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00100000:1.0:1787784788.160419:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784788.161269:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658155776 00000100:00100000:2.0:1787784788.161280:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658155776 00000100:00100000:2.0:1787784788.161335:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1136 00000100:00100000:2.0:1787784788.161345:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29388:x1874627658155776:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784788.161445:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29388:x1874627658155776:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 101us (1035us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784788.161452:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1136 00000100:00100000:1.0:1787784788.201745:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.203315:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658155904 00000100:00100000:0.0:1787784788.203328:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658155904 00002000:00100000:0.0:1787784788.203377:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x16:0x0] (0->4095) req@ffff989401cc0f00 x1874627658155904/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:669/0 lens 488/0 e 0 to 0 dl 1787784799 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784788.203387:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cc0f00 with x1874627658155904 ext(0->4095) 00002000:00010000:0.0:1787784788.203392:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989401cc0f00 00000100:00100000:0.0:1787784788.203438:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 101 00000100:00100000:0.0:1787784788.203447:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658155904:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1787784788.203819:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658155904:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 371us (2084us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784788.203826:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 101 00002000:00100000:0.0:1787784788.203832:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x16:0x0] (0->4095) req@ffff989401cc0f00 x1874627658155904/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:669/0 lens 488/4536 e 0 to 0 dl 1787784799 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784788.203841:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cc0f00 with x1874627658155904 ext(0->4095) 00002000:00010000:0.0:1787784788.203851:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989401cc0f00 00000100:00100000:1.0:1787784788.215776:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.216000:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658156032 00000100:00100000:0.0:1787784788.216010:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658156032 00000100:00100000:0.0:1787784788.216066:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1137 00000100:00100000:0.0:1787784788.216076:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29388:x1874627658156032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784788.216095:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784788.216114:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65a3 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1187 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.216173:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.216187:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.216198:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.216296:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65a3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.216313:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.216322:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:0.0:1787784788.216325:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784788.216329:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2a00, rc 0) 00000100:00100000:0.0:1787784788.216388:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29388:x1874627658156032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 313us (621us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784788.216405:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1137 00000100:00100000:1.0:1787784788.233065:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.233357:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658156160 00000100:00100000:3.0:1787784788.233367:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658156160 00000100:00100000:3.0:1787784788.233430:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1138 00000100:00100000:3.0:1787784788.233441:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29388:x1874627658156160:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784788.233779:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784788.234607:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29388:x1874627658156160:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 1164us (1547us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784788.234616:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1138 00000100:00100000:1.0:1787784788.260207:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.260680:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658156288 00000100:00100000:3.0:1787784788.260690:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658156288 00000100:00100000:3.0:1787784788.260779:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 307 00000100:00100000:3.0:1787784788.260789:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658156288:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784788.260829:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261083:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784788.261088:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784788.261095:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261109:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261140:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261241:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261250:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261256:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784788.261261:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261270:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261292:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.261458:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65b1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784788.264668:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658156288:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 3870us (4460us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784788.264682:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 307 00000100:00100000:2.0:1787784788.274180:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.274267:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658156544 00000100:00100000:0.0:1787784788.274277:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658156544 00000100:00100000:0.0:1787784788.274332:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1139 00000100:00100000:0.0:1787784788.274343:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29388:x1874627658156544:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784788.274428:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.274446:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.274455:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.274487:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784788.274501:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 49 pid: 10850 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784788.274516:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668408320:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784788.274610:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ebd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784788.277296:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.277444:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668408320:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784788.277471:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784788.277654:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658156672 00000100:00100000:3.0:1787784788.277661:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658156672 00010000:00010000:3.0:1787784788.277695:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 50 pid: 10850 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784788.277729:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:3.0:1787784788.277738:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2357:x1874627658156672:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784788.277753:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784788.277759:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 50 pid: 10850 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.277771:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 50 pid: 10850 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.277793:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc65aa lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1187 expref: 50 pid: 10850 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.277821:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:3.0:1787784788.277826:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.277834:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.277851:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.277877:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784788.277922:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658156672:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 184us (634us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784788.277929:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 340 00010000:00010000:2.0:1787784788.278080:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278088:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278348:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278357:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278364:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278370:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784788.278374:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278382:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278391:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:2.0:1787784788.278394:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784788.278398:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784788.278404:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc65b8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784788.278543:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29388:x1874627658156544:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 4201us (4373us total) trans 4294968039 rc 0/0 00000100:00100000:2.0:1787784788.278552:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1139 00000040:00080000:3.0:1787784788.278912:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784788.278917:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784788.279762:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:3.0:1787784788.279772:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784788.285624:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784788.286965:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658156928 00000100:00100000:0.0:1787784788.286986:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658156928 00000100:00100000:0.0:1787784788.287274:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 224 00000100:00100000:0.0:1787784788.287286:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29388:x1874627658156928:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784788.287526:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29388:x1874627658156928:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 243us (1910us total) trans 4294968040 rc 0/0 00000100:00100000:0.0:1787784788.287535:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 224 00000040:00080000:3.0:1787784788.891233:0:20536:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784788.891245:0:20536:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784788.891290:0:20536:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 32 count 3 00000040:00080000:3.0:1787784788.891296:0:20536:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784788.917288:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784788.994450:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784788.995244:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658157056 00000100:00100000:3.0:1787784788.995254:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658157056 00000100:00100000:3.0:1787784788.995305:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 308 00000100:00100000:3.0:1787784788.995314:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658157056:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784788.995341:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784788.995358:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b118e expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.995383:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784788.995387:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784788.995396:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 16 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.995417:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 16 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784788.995430:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784788.996032:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658157056:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 717us (1587us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784788.996039:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 308 00000100:00100000:3.0:1787784789.018184:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784789.018223:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658157312 00000100:00100000:3.0:1787784789.018233:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658157312 00000100:00100000:3.0:1787784789.018297:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1140 00000100:00100000:3.0:1787784789.018308:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:29391:x1874627658157312:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784789.018326:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784789.018345:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65c6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1195 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.018440:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65cd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.018466:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.019234:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784789.019251:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:3.0:1787784789.019458:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65c6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1195 expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.020820:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65cd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1195 expref: 99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.020835:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:3.0:1787784789.020838:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1787784789.020842:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784789.020912:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:29391:x1874627658157312:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2605us (2737us total) trans 4294968041 rc 0/0 00000100:00100000:3.0:1787784789.020922:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1140 00000100:00100000:2.0:1787784789.026874:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784789.028440:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658157440 00000100:00100000:1.0:1787784789.028451:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658157440 00000100:00100000:1.0:1787784789.028514:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 309 00000100:00100000:1.0:1787784789.028525:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658157440:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:1.0:1787784789.028579:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784789.028599:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784789.028609:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 16 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784789.028629:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784789.028634:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784789.028639:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 5/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784789.028671:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 16 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784789.028685:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 16 pid: 13825 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784789.028697:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668408448:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784789.028801:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784789.028811:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 10 pid: 6670 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784789.028820:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668408576:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784789.028843:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f900 going to sleep for 11 seconds 00000100:00080000:1.0:1787784789.028870:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784789.028874:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784789.038014:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784789.038047:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784789.038097:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668408448:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784789.038109:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668408576:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784789.038178:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784789.038208:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658157696 00000100:00100000:2.0:1787784789.038214:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658157696 00010000:00010000:2.0:1787784789.038257:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 11 pid: 6670 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784789.038284:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658157568 00000100:00100000:2.0:1787784789.038286:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658157568 00010000:00010000:2.0:1787784789.038298:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 17 pid: 13825 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784789.038321:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:2.0:1787784789.038329:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658157696:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784789.038343:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784789.038363:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 11 pid: 6670 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.038374:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 11 pid: 6670 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.038397:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc659c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1180 expref: 11 pid: 6670 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.038412:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784789.038455:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658157696:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 126us (451us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784789.038461:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:2.0:1787784789.039348:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 342 00000100:00100000:2.0:1787784789.039364:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658157568:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784789.039376:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784789.039391:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 17 pid: 13825 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.039405:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 17 pid: 13825 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.039421:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc65bf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b118e expref: 17 pid: 13825 timeout: 680 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.039434:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784789.039438:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784789.039443:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.039453:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784789.039634:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784789.039716:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658157568:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 312us (1623us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784789.039726:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 342 00000100:00080000:2.0:1787784789.039844:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787784789.040304:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040315:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040339:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040347:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040356:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040363:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784789.040368:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040377:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040403:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.040411:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc65db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784789.040457:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658157440:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 11935us (13591us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784789.040465:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 309 00000100:00100000:3.0:1787784789.052347:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784789.052383:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658157952 00000100:00100000:3.0:1787784789.052393:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658157952 00000100:00100000:3.0:1787784789.052439:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 225 00000100:00100000:3.0:1787784789.052448:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:29391:x1874627658157952:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:3.0:1787784789.053356:0:5814:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x20:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65cd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1195 expref: 99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784789.053609:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968042 00000001:00080000:3.0:1787784789.054327:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968043 00010000:00010000:3.0:1787784789.054385:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.054402:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 49 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.054412:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.054420:0:5814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.054454:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.054470:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 102 pid: 5813 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784789.054484:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668408704:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784789.054590:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 49 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.054599:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 49 pid: 5812 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784789.054621:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45c780 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668408832:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784789.054648:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5540 going to sleep for 11 seconds 00000100:00100000:3.0:1787784789.061925:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45d2c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668408704:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784789.061944:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45c780 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668408832:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784789.061970:0:5814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784789.063038:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784789.063071:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784789.063143:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658158208 00000100:00100000:0.0:1787784789.063173:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658158208 00010000:00010000:0.0:1787784789.063229:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 50 pid: 5812 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784789.063255:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658158080 00000100:00100000:0.0:1787784789.063258:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658158080 00010000:00010000:0.0:1787784789.063272:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 103 pid: 5813 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784789.063290:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 343 00000100:00100000:0.0:1787784789.063299:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2356:x1874627658158208:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784789.063314:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784789.063320:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 50 pid: 5812 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063331:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 50 pid: 5812 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063361:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc652c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1148 expref: 50 pid: 5812 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063374:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:0.0:1787784789.063379:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063386:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063393:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784789.063435:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2356:x1874627658158208:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 138us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784789.063442:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 343 00000100:00100000:0.0:1787784789.063464:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 344 00000100:00100000:0.0:1787784789.063470:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:2356:x1874627658158080:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784789.063493:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784789.063499:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 103 pid: 5813 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063509:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 103 pid: 5813 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063521:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc655d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b115d expref: 103 pid: 5813 timeout: 680 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063543:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:0.0:1787784789.063547:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063554:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063570:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784789.063590:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784789.063609:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:2356:x1874627658158080:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 139us (537us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784789.063616:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 344 00010000:00010000:3.0:1787784789.063781:0:5814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.063790:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784789.063902:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968044 00000001:00080000:3.0:1787784789.064093:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968045 00000001:00080000:3.0:1787784789.064173:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968046 00000100:00100000:3.0:1787784789.064251:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668408960:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1787784789.064293:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784789.064320:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5840 going to sleep for 11 seconds 00000100:00080000:3.0:1787784789.064337:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784789.064802:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668408960 00000100:00100000:0.0:1787784789.064808:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668408960 00000100:00100000:0.0:1787784789.064853:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:0.0:1787784789.064860:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668408960:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1787784789.064927:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668408960:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 67us (632us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784789.064934:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:3.0:1787784789.065406:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668408960:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:3.0:1787784789.065721:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968047 00000100:00100000:3.0:1787784789.065782:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668409088:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1787784789.065824:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784789.065853:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5540 going to sleep for 11 seconds 00000100:00080000:3.0:1787784789.065871:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784789.066688:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668409088 00000100:00100000:0.0:1787784789.066695:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668409088 00000100:00100000:0.0:1787784789.066742:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:0.0:1787784789.066751:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668409088:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1787784789.066905:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5814:x1874627668409088:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 155us (1080us total) trans 4294967355 rc 0/0 00000100:00100000:0.0:1787784789.066923:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00000100:00100000:3.0:1787784789.067338:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668409088:0@lo:1000:mdt_rdpg00_000.0 00010000:00010000:3.0:1787784789.067360:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.067370:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.067377:0:5814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.067383:0:5814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784789.067387:0:5814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.067394:0:5814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.067409:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:3.0:1787784789.067411:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:3.0:1787784789.067416:0:5814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.067422:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65e2 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784789.067612:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x47:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787784789.068010:0:5814:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].9, 64 off8792 00000004:00080000:3.0:1787784789.068023:0:5814:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 580154874805ns, next at 581154872350ns, rc = 0 00000001:00080000:3.0:1787784789.068038:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968048 00010000:00010000:3.0:1787784789.068076:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5100) 00010000:00010000:3.0:1787784789.068080:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1787784789.068087:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65cd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1195 expref: 106 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784789.068538:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:29391:x1874627658157952:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 16090us (16200us total) trans 4294968049 rc 0/0 00000100:00100000:3.0:1787784789.068548:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 225 00000020:00080000:3.0:1787784789.068551:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000001:00080000:2.0:1787784789.104423:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968040 is committed 00000001:00080000:2.0:1787784789.104446:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968035 is committed 00000004:00080000:3.0:1787784789.105203:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 17 committed 00000020:00080000:3.0:1787784789.105413:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 22 batchid updated 00000020:00080000:3.0:1787784789.105429:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 22 00000020:00080000:3.0:1787784789.105439:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 22 00000100:00100000:3.0:1787784789.105508:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403597840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668409216:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1787784789.105585:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784789.105715:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668409216 00000100:00100000:3.0:1787784789.105722:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668409216 00000100:00100000:3.0:1787784789.105772:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:3.0:1787784789.105780:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668409216:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1787784789.105805:0:6671:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:24 count 1 00002000:00010000:3.0:1787784789.105836:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x18:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6a8f6c28, ctime: 0x6a8f6e48, blocks: 0xb 00010000:00010000:3.0:1787784789.105844:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.105860:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=580s v=(0 0) 00010000:00010000:3.0:1787784789.105865:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784789.105871:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.105892:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.105902:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.105910:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.105927:0:6671:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784789.106792:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668409216:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1010us (1204us total) trans 4294967366 rc 0/0 00000100:00100000:3.0:1787784789.106802:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00080000:3.0:1787784789.106826:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787784789.106839:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784789.106846:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.106857:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784789.106862:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.106872:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784789.106875:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989403597840/2, rc 0, transno 4294967366 00000100:00100000:2.0:1787784789.106885:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403597840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668409216:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1787784789.106910:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.106918:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc65e9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784789.106935:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784789.106937:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784789.106939:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784789.489482:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668409344:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784789.489569:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784789.489695:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668409344 00000100:00100000:1.0:1787784789.489702:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668409344 00000100:00100000:1.0:1787784789.489794:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1787784789.489811:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668409344:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784789.489849:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784789.489895:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668409344:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784789.489903:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1787784789.490008:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668409344:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784789.490455:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668409728:0@lo:400:kworker.0 00000100:00100000:3.0:1787784789.490490:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784789.490528:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784789.490534:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668409856:0@lo:400:kworker.0 00000100:00100000:3.0:1787784789.490567:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784789.490634:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668409600:0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.490672:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784789.490698:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784789.490704:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668409472:0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.490732:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784789.490765:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668409728 00000100:00100000:2.0:1787784789.490769:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668409728 00000100:00100000:2.0:1787784789.490808:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668409856 00000100:00100000:2.0:1787784789.490811:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668409856 00000100:00100000:2.0:1787784789.490829:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668409600 00000100:00100000:2.0:1787784789.490831:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668409600 00000100:00100000:2.0:1787784789.490844:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1787784789.490852:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668409728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.490890:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668409728:12345-0@lo:400:kworker.0 Request processed in 39us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784789.490897:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1787784789.490909:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:2.0:1787784789.490915:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668409856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.490952:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668409856:12345-0@lo:400:kworker.0 Request processed in 38us (394us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784789.490957:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:2.0:1787784789.490968:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:2.0:1787784789.490974:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668409600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.491004:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668409600:12345-0@lo:400:kworker.0 Request processed in 30us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784789.491010:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:2.0:1787784789.491027:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668409472 00000100:00100000:2.0:1787784789.491030:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668409472 00000100:00100000:2.0:1787784789.491054:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00100000:2.0:1787784789.491060:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668409472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.491089:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668409472:12345-0@lo:400:kworker.0 Request processed in 29us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784789.491312:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00080000:2.0:1787784789.491340:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784789.491370:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035961c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668409472:0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.491382:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668409600:0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.491412:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668409856:0@lo:400:kworker.0 00000100:00100000:2.0:1787784789.491420:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403596940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668409728:0@lo:400:kworker.0 00010000:00010000:1.0:1787784789.954954:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784790.094622:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967355 is committed 00000001:00080000:0.0:1787784790.094641:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967354 is committed 00010000:00010000:0.0:1787784790.963493:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784791.425299:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784791.425426:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658158336 00000100:00100000:3.0:1787784791.425436:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658158336 00000100:00100000:3.0:1787784791.425492:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1141 00000100:00100000:3.0:1787784791.425503:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:29409:x1874627658158336:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:3.0:1787784791.425525:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784791.425559:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b119c expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.425626:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.425640:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.425662:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.425677:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.425684:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.425690:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784791.425856:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784791.426291:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6605 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426301:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6605 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426308:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6605 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784791.426341:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:3.0:1787784791.426345:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784791.426364:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6605 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426375:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6605 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426383:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426399:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426406:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426412:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784791.426416:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426423:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426435:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:3.0:1787784791.426438:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784791.426442:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426449:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc65f7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426461:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426467:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426473:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426478:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784791.426487:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426494:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426502:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff9895198b5400) 00010000:00010000:3.0:1787784791.426505:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784791.426507:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426513:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc65fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426670:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc65f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b119c expref: 106 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426688:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6605 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b119c expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.426695:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5500) 00010000:00010000:3.0:1787784791.426697:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784791.426699:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784791.426991:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:29409:x1874627658158336:12345-192.168.203.43@tcp:101:diff.0 Request processed in 1488us (1851us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784791.426999:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1141 00000100:00100000:0.0:1787784791.442793:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784791.442865:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658158592 00000100:00100000:2.0:1787784791.442874:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658158592 00000100:00100000:2.0:1787784791.442931:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1142 00000100:00100000:2.0:1787784791.442941:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:29409:x1874627658158592:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:2.0:1787784791.442961:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784791.442980:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc660c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11a3 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784791.443079:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6613 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784791.443092:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6613 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784791.443103:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6613 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784791.443118:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:2.0:1787784791.443275:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc660c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11a3 expref: 108 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784791.443293:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6613 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11a3 expref: 107 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784791.443301:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5500) 00010000:00010000:2.0:1787784791.443303:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784791.443307:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784791.443356:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:29409:x1874627658158592:12345-192.168.203.43@tcp:101:diff.0 Request processed in 416us (573us total) trans 4294968050 rc 0/0 00000100:00100000:2.0:1787784791.443363:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1142 00000100:00100000:1.0:1787784791.447723:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784791.447921:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658158720 00000100:00100000:2.0:1787784791.447931:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658158720 00000100:00100000:2.0:1787784791.447995:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 49 00000100:00100000:2.0:1787784791.448006:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2357:x1874627658158720:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784791.448021:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784791.448045:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6613 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11a3 expref: 107 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784791.448063:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5500) 00010000:00010000:2.0:1787784791.448068:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784791.448124:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658158720:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 121us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784791.448140:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 49 00000100:00100000:1.0:1787784791.456668:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784791.457276:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658158976 00000100:00100000:3.0:1787784791.457286:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658158976 00000100:00100000:3.0:1787784791.457378:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 50 00000100:00100000:3.0:1787784791.457388:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:29409:x1874627658158976:12345-192.168.203.43@tcp:103:diff.0 00010000:00010000:3.0:1787784791.457410:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784791.457434:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6605 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b119c expref: 106 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.457452:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5500) 00010000:00010000:3.0:1787784791.457461:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784791.457509:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:29409:x1874627658158976:12345-192.168.203.43@tcp:103:diff.0 Request processed in 122us (847us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784791.457518:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 50 00000100:00100000:3.0:1787784791.460620:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784791.460661:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658159104 00000100:00100000:3.0:1787784791.460829:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658159104 00000100:00100000:3.0:1787784791.460880:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1143 00000100:00100000:3.0:1787784791.460890:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:29409:x1874627658159104:12345-192.168.203.43@tcp:58:diff.0 00010000:00010000:3.0:1787784791.460985:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.461001:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.461013:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784791.461297:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].32, 136 off12544 00000100:00100000:3.0:1787784791.461371:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:29409:x1874627658159104:12345-192.168.203.43@tcp:58:diff.0 Request processed in 482us (758us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784791.461378:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1143 00000100:00100000:3.0:1787784791.463619:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784791.463682:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658159232 00000100:00100000:3.0:1787784791.463690:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658159232 00000100:00100000:3.0:1787784791.463746:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1144 00000100:00100000:3.0:1787784791.463765:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:29409:x1874627658159232:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:3.0:1787784791.463785:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784791.463805:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11aa expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.463855:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.463866:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 106 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.463881:0:10850:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.463895:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 106 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.463908:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:3.0:1787784791.463911:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784791.463915:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 106 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784791.463922:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7400, rc 0) 00000100:00100000:3.0:1787784791.463962:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:29409:x1874627658159232:12345-192.168.203.43@tcp:101:diff.0 Request processed in 208us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784791.463977:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1144 00010000:00010000:3.0:1787784791.986594:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784792.369394:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784792.369405:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784792.370311:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 3 00000040:00080000:0.0:1787784792.370322:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784792.370443:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ae580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668409984:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784792.370569:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f000 going to sleep for 11 seconds 00000100:00100000:0.0:1787784792.376583:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ae580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668409984:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784792.384630:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.386185:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658159360 00000100:00100000:2.0:1787784792.386198:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658159360 00000100:00100000:2.0:1787784792.386286:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1145 00000100:00100000:2.0:1787784792.386296:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cefc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29410:x1874627658159360:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784792.386331:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784792.386441:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cefc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29410:x1874627658159360:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 146us (1819us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.386449:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1145 00000100:00080000:1.0:1787784792.388054:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784792.399089:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.400687:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658159488 00000100:00100000:1.0:1787784792.400700:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658159488 00000100:00100000:1.0:1787784792.400764:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1146 00000100:00100000:1.0:1787784792.400775:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29410:x1874627658159488:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1787784792.400845:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400863:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400875:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400907:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400915:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400938:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400960:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400969:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.400976:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784792.401024:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:1.0:1787784792.401029:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784792.401042:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401053:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401059:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401067:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401072:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784792.401076:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401084:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401103:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784792.401106:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784792.401111:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401118:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc662f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401141:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401195:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401201:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401206:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784792.401209:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401215:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401222:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x329c2aeb (ffff98950a38cf00) 00010000:00010000:1.0:1787784792.401225:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784792.401228:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401234:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc6636 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401259:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401265:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401271:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401276:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784792.401279:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401285:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401292:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:1.0:1787784792.401294:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784792.401297:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.401303:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc663d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784792.401381:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29410:x1874627658159488:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 606us (2300us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.401388:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1146 00000100:00100000:0.0:1787784792.405401:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.407048:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658159744 00000100:00100000:2.0:1787784792.407061:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658159744 00000100:00100000:2.0:1787784792.407127:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1147 00000100:00100000:2.0:1787784792.407137:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29410:x1874627658159744:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784792.407192:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784792.407213:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6644 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11b1 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407461:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407476:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407488:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784792.407498:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:2.0:1787784792.407607:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407619:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407625:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407631:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784792.407635:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407642:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407653:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:2.0:1787784792.407655:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784792.407659:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407665:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc664b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407705:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6644 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11b1 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407715:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784792.407717:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784792.407721:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc6644 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11b1 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.407729:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784792.408414:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29410:x1874627658159744:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1275us (3017us total) trans 4294968051 rc 301/301 00000100:00100000:2.0:1787784792.408422:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1147 00000100:00100000:2.0:1787784792.421437:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.421808:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658159872 00000100:00100000:3.0:1787784792.421818:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658159872 00000100:00100000:3.0:1787784792.421925:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1148 00000100:00100000:3.0:1787784792.421936:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29410:x1874627658159872:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784792.422002:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784792.422106:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29410:x1874627658159872:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 172us (677us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.422114:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1148 00000100:00100000:1.0:1787784792.433290:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.433676:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658160128 00000100:00100000:3.0:1787784792.433685:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658160128 00000100:00100000:3.0:1787784792.433748:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1149 00000100:00100000:3.0:1787784792.433759:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29410:x1874627658160128:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784792.433902:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784792.433925:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6659 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11b8 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.438192:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6660 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.438212:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6660 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.438224:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6660 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784792.438250:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x21:0x0]: returning size 0 00010000:00010000:3.0:1787784792.438870:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6659 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11b8 expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.438892:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6660 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11b8 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.438901:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5200) 00010000:00010000:3.0:1787784792.438904:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784792.438907:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784792.439044:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29410:x1874627658160128:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5286us (5762us total) trans 4294968052 rc 0/0 00000100:00100000:3.0:1787784792.439054:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1149 00000100:00100000:0.0:1787784792.446368:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.446550:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658160384 00000100:00100000:2.0:1787784792.446559:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658160384 00000100:00100000:2.0:1787784792.446617:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1150 00000100:00100000:2.0:1787784792.446627:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29410:x1874627658160384:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784792.446682:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784792.446707:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6660 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11b8 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.446727:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5200) 00010000:00010000:2.0:1787784792.446738:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784792.446757:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.446769:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 105 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.446776:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 4/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.446809:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 105 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.446821:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 105 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.446838:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668410112:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784792.446917:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507f859c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784792.459430:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668410112:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784792.459459:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.476360:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.476609:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658160512 00000100:00100000:2.0:1787784792.476619:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658160512 00010000:00010000:2.0:1787784792.476656:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 106 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.476688:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 345 00000100:00100000:2.0:1787784792.476696:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658160512:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784792.476709:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784792.476716:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 106 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.476728:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 106 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.476749:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6572 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b116b expref: 106 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.476763:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:2.0:1787784792.476768:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.476788:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.476801:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.476826:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784792.476874:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:2357:x1874627658160512:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 178us (522us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.476882:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 345 00010000:00010000:0.0:1787784792.477390:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477400:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477423:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477445:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477453:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477692:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477700:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477706:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477712:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784792.477716:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477887:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477898:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784792.477901:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784792.477904:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477911:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc666e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477922:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477933:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477940:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.477945:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784792.477948:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.478013:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.478022:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0xa79405c7 (ffff989503a6d200) 00010000:00010000:0.0:1787784792.478025:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784792.478028:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.478034:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6675 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0xa79405c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.478251:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29410:x1874627658160384:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 31624us (31891us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784792.478260:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1150 00000100:00100000:0.0:1787784792.487807:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.491970:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658160640 00000100:00100000:3.0:1787784792.491985:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658160640 00000100:00100000:3.0:1787784792.492060:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1151 00000100:00100000:3.0:1787784792.492070:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29410:x1874627658160640:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784792.492089:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784792.492113:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc667c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11bf expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.492192:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.492206:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.492234:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.492337:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc667c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.492356:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.492365:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:3.0:1787784792.492368:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784792.492371:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784792.492439:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29410:x1874627658160640:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 370us (4639us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.492446:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1151 00000100:00100000:2.0:1787784792.496318:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.496456:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658160896 00000100:00100000:3.0:1787784792.496465:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658160896 00000100:00100000:3.0:1787784792.496521:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1152 00000100:00100000:3.0:1787784792.496530:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29410:x1874627658160896:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784792.496620:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.496635:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.496646:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784792.496878:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e6b434c4 reply state 00000000c1f4542b transno 4294968053 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784792.496889:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784792.497449:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29410:x1874627658160896:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 917us (1137us total) trans 4294968053 rc 0/0 00000100:00100000:2.0:1787784792.497458:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1152 00000100:00100000:1.0:1787784792.500660:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:0.0:1787784792.500753:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.500766:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.500796:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.500803:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784792.500808:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.500816:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.500834:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:0.0:1787784792.500837:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784792.500842:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.500849:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc668a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.502055:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658161024 00000100:00100000:2.0:1787784792.502067:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658161024 00000100:00100000:2.0:1787784792.502134:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1153 00000100:00100000:2.0:1787784792.502175:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29410:x1874627658161024:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784792.502200:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784792.502219:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11c6 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.502740:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.502756:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.502764:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:2.0:1787784792.502767:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784792.502770:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:2.0:1787784792.502852:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29410:x1874627658161024:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 709us (2200us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.502859:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1153 00000100:00100000:0.0:1787784792.509284:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.509570:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658161152 00000100:00100000:3.0:1787784792.509579:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658161152 00000100:00100000:3.0:1787784792.509641:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 226 00000100:00100000:3.0:1787784792.509651:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29410:x1874627658161152:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1787784792.509781:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29410:x1874627658161152:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 131us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.509788:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 226 00000100:00100000:0.0:1787784792.515625:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.515773:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658161408 00000100:00100000:3.0:1787784792.515792:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658161408 00000100:00100000:3.0:1787784792.515847:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 227 00000100:00100000:3.0:1787784792.515857:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d6c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29410:x1874627658161408:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784792.516117:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d6c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29410:x1874627658161408:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 260us (500us total) trans 4294968054 rc 0/0 00000100:00100000:3.0:1787784792.516126:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 227 00000100:00100000:1.0:1787784792.521866:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.521972:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658161664 00000100:00100000:2.0:1787784792.521982:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658161664 00000100:00100000:2.0:1787784792.522035:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1154 00000100:00100000:2.0:1787784792.522045:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29410:x1874627658161664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784792.522200:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29410:x1874627658161664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 156us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.522208:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1154 00000100:00100000:1.0:1787784792.531995:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.532386:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658161920 00000100:00100000:2.0:1787784792.532398:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658161920 00000100:00100000:2.0:1787784792.532456:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1155 00000100:00100000:2.0:1787784792.532467:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29410:x1874627658161920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784792.532489:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784792.532509:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6698 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11cd expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.532620:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.532636:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.532645:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 4/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.532677:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.532691:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 51 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.532704:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391283c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668410240:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784792.532788:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507f850c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784792.537040:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.537228:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658162048 00000100:00100000:1.0:1787784792.537237:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658162048 00010000:00010000:1.0:1787784792.537288:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 52 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784792.537330:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 346 00000100:00100000:1.0:1787784792.537341:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:2356:x1874627658162048:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784792.537356:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784792.537363:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 52 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.537376:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 52 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.537398:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:1.0:1787784792.537403:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.537412:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.537420:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.537439:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784792.537492:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:2356:x1874627658162048:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 153us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.537500:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 346 00010000:00010000:2.0:1787784792.538914:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6691 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11c6 expref: 51 pid: 5811 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.538935:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391283c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668410240:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784792.538967:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.538976:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.538981:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784792.538999:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x21:0x0]: returning size 0 00000004:00080000:2.0:1787784792.539566:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x19:0x0], index 1: 0 00010000:00010000:2.0:1787784792.539700:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.539709:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.539722:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.539728:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784792.539731:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.539739:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.539753:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:2.0:1787784792.539756:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784792.539760:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.539766:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc669f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.540190:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6698 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11cd expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.540202:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:2.0:1787784792.540205:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784792.540209:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6698 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11cd expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.540219:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784792.540376:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29410:x1874627658161920:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 7910us (8390us total) trans 4294968055 rc 301/301 00000100:00100000:2.0:1787784792.540388:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1155 00000004:00080000:2.0:1787784792.561665:0:6996:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 583648514988ns, next at 584648512510ns, rc = 0 00000100:00100000:3.0:1787784792.561732:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784792.561756:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391283c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668410368:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784792.561833:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784792.562203:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.562275:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668410368 00000100:00100000:2.0:1787784792.562283:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668410368 00000100:00100000:2.0:1787784792.562344:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:2.0:1787784792.562353:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668410368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784792.562388:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784792.562432:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668410368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (599us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.562438:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00080000:0.0:1787784792.562501:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784792.562933:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967355 gen 2 00000004:00080000:3.0:1787784792.562940:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967355 committed transno 4294967355 00000100:00100000:3.0:1787784792.562975:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391283c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668410368:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784792.562998:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658162304 00000100:00100000:3.0:1787784792.563014:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658162304 00000100:00100000:3.0:1787784792.563053:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 228 00000100:00100000:3.0:1787784792.563060:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29410:x1874627658162304:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784792.563671:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:29410:x1874627658162304:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 609us (1468us total) trans 4294968056 rc 0/0 00000100:00100000:3.0:1787784792.563680:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 228 00000100:00100000:0.0:1787784792.574524:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.574626:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658162432 00000100:00100000:3.0:1787784792.574636:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658162432 00000100:00100000:3.0:1787784792.574692:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1156 00000100:00100000:3.0:1787784792.574703:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508341850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:29410:x1874627658162432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784792.574728:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784792.574748:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66ad lrc: 2/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11d4 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.574943:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66b4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.575165:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66b4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.575180:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66b4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784792.575218:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x21:0x0]: returning size 0 00000004:00200000:3.0:1787784792.575222:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784792.575241:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66b4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.575894:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66ad lrc: 0/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11d4 expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.575931:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66b4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11d4 expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.575940:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:3.0:1787784792.575943:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1787784792.575947:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784792.576137:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508341850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29410:x1874627658162432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1435us (1621us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.576185:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1156 00000100:00100000:0.0:1787784792.593649:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784792.593685:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658162560 00000100:00100000:0.0:1787784792.593693:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658162560 00000100:00100000:0.0:1787784792.593757:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 311 00000100:00100000:0.0:1787784792.593768:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658162560:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784792.593798:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784792.593817:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11db expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784792.593840:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x19:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784792.593859:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=583s v=(0 0) 00010000:00010000:0.0:1787784792.593863:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784792.593872:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.593905:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.593917:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:0.0:1787784792.593962:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658162560:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 197us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784792.593969:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 311 00000100:00080000:1.0:1787784792.596450:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784792.622704:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.622834:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658162816 00000100:00100000:3.0:1787784792.622844:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658162816 00000100:00100000:3.0:1787784792.622909:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1157 00000100:00100000:3.0:1787784792.622920:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29410:x1874627658162816:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784792.623035:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29410:x1874627658162816:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 117us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.623043:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1157 00000100:00100000:3.0:1787784792.627037:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.627577:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658162944 00000100:00100000:2.0:1787784792.627588:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658162944 00000100:00100000:2.0:1787784792.627640:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 312 00000100:00100000:2.0:1787784792.627650:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:29410:x1874627658162944:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784792.627669:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784792.627688:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11e2 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.627718:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.627727:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784792.627731:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784792.627736:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.627773:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 12 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.627786:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 12 pid: 6670 timeout: 683 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.627800:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668410496:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784792.627885:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507f859c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784792.649035:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784792.649221:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658163072 00000100:00100000:0.0:1787784792.649230:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658163072 00010000:00010000:0.0:1787784792.649280:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 13 pid: 6670 timeout: 683 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.649320:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:0.0:1787784792.649329:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658163072:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784792.649363:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784792.649384:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 13 pid: 6670 timeout: 683 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.649399:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 13 pid: 6670 timeout: 683 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.649422:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784792.649427:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784792.649435:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.649445:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.649491:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.649504:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035acb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668410624:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784792.649599:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f000 going to sleep for 11 seconds 00010000:00010000:2.0:1787784792.650721:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488b76e5/0xb0ce63137acc66bb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11db expref: 13 pid: 6670 timeout: 683 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.650741:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668410496:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784792.650763:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.650773:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784792.650833:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:29410:x1874627658162944:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 23185us (23805us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.650842:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 312 00000100:00100000:2.0:1787784792.653013:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035acb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668410624:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784792.653036:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784792.653084:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658163072:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 3756us (4059us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.653106:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:3.0:1787784792.668751:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.668868:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658163200 00000100:00100000:1.0:1787784792.668878:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658163200 00002000:00100000:1.0:1787784792.668939:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x19:0x0] (0->157) req@ffff989401cefc00 x1874627658163200/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:673/0 lens 648/0 e 0 to 0 dl 1787784803 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784792.668951:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cefc00 with x1874627658163200 ext(0->157) 00002000:00010000:1.0:1787784792.668957:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401cefc00 00000100:00100000:1.0:1787784792.668998:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 102 00000100:00100000:1.0:1787784792.669008:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cefc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658163200:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1787784792.669547:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cefc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658163200:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 528us (792us total) trans 4294967367 rc 0/0 00000100:00100000:1.0:1787784792.669557:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 102 00002000:00100000:1.0:1787784792.669564:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x19:0x0] (0->157) req@ffff989401cefc00 x1874627658163200/t4294967367(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:673/0 lens 648/448 e 0 to 0 dl 1787784803 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784792.669573:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cefc00 with x1874627658163200 ext(0->157) 00002000:00010000:1.0:1787784792.669576:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401cefc00 00000100:00100000:0.0:1787784792.690940:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.691441:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658163328 00000100:00100000:2.0:1787784792.691449:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658163328 00000100:00100000:2.0:1787784792.691505:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1158 00000100:00100000:2.0:1787784792.691516:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29410:x1874627658163328:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000004:00080000:3.0:1787784792.782770:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 11 committed 00000001:00080000:3.0:1787784792.782788:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968055 is committed 00000001:00080000:3.0:1787784792.782795:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968050 is committed 00000001:00080000:3.0:1787784792.782799:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968056 is committed 00000004:00080000:3.0:1787784792.782810:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 10 committed 00000020:00080000:3.0:1787784792.782818:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9895401a5840 refcount 2 committed 0 result 0 batchid 23 00000020:00080000:3.0:1787784792.782836:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989401288500 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9895401a5780 00000020:00080000:3.0:1787784792.782839:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989401288100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9894019d5800 00000020:00080000:3.0:1787784792.782991:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 23 committed batchid 22 00000100:00100000:1.0:1787784792.783496:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29410:x1874627658163328:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 91976us (92560us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.783507:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1158 00000100:00100000:1.0:1787784792.783549:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784792.783563:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668410752:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787784792.783614:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:3.0:1787784792.785962:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 22 00000100:00100000:3.0:1787784792.786008:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668410752 00000100:00100000:3.0:1787784792.786017:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668410752 00000100:00100000:3.0:1787784792.786065:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1787784792.786075:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668410752:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787784792.786115:0:13825:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:22 count 1 00002000:00010000:3.0:1787784792.786171:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 158, mtime: 0x6a8f6e53, atime: 0x0, ctime: 0x6a8f6e53, blocks: 0xb 00010000:00010000:3.0:1787784792.786180:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.786196:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=583s v=(0 0) 00010000:00010000:3.0:1787784792.786201:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784792.786207:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.786222:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.786231:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.786238:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.786383:0:13825:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784792.787074:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668410752:12345-0@lo:6:osp-syn-0-0.0 Request processed in 998us (3457us total) trans 4294967356 rc 0/0 00000100:00100000:3.0:1787784792.787084:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00010000:00010000:3.0:1787784792.787134:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784792.787140:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.787170:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784792.787175:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.787184:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.787200:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.787207:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc66c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.787226:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784792.787228:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784792.787229:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1787784792.787872:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401cecb40/2, rc 0, transno 4294967356 00000100:00100000:3.0:1787784792.787882:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668410752:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784792.789861:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784792.790141:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658163456 00000100:00100000:0.0:1787784792.790181:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658163456 00000100:00100000:0.0:1787784792.790227:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 314 00000100:00100000:0.0:1787784792.790236:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2354:x1874627658163456:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1787784792.836065:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2354:x1874627658163456:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 45820us (46201us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.836079:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 314 00000001:00080000:2.0:1787784792.839902:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967367 is committed 00000001:00080000:2.0:1787784792.839928:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967366 is committed 00000100:00100000:1.0:1787784792.840971:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.841017:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.841049:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.841082:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.841127:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.841172:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.841198:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658163840 00000100:00100000:1.0:1787784792.841206:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658163840 00000100:00100000:1.0:1787784792.841248:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:1.0:1787784792.841256:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658163840:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784792.841333:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658163840:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.841341:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:1.0:1787784792.841372:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658164224 00000100:00100000:1.0:1787784792.841376:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658164224 00000100:00100000:1.0:1787784792.841406:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658164096 00000100:00100000:1.0:1787784792.841409:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658164096 00000100:00100000:1.0:1787784792.841426:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658164352 00000100:00100000:1.0:1787784792.841429:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658164352 00000100:00100000:1.0:1787784792.841443:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 350 00000100:00100000:1.0:1787784792.841449:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658164224:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784792.841490:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658164224:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 41us (440us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.841497:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 350 00000100:00100000:1.0:1787784792.841509:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 351 00000100:00100000:1.0:1787784792.841515:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658164096:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784792.841560:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658164096:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 45us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.841580:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 351 00000100:00100000:3.0:1787784792.841810:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658163712 00000100:00100000:3.0:1787784792.841815:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658163712 00000100:00100000:3.0:1787784792.841841:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658163968 00000100:00100000:3.0:1787784792.841844:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658163968 00000100:00100000:3.0:1787784792.841866:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:3.0:1787784792.841873:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658163968:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784792.842018:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658163968:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 144us (999us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.842026:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:3.0:1787784792.842041:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1159 00000100:00100000:3.0:1787784792.842050:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29410:x1874627658163712:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784792.842091:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784792.842116:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66b4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11d4 expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.842136:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:3.0:1787784792.842140:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784792.842198:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.842212:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.842223:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784792.842407:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e70d2859 reply state 00000000c1f4542b transno 4294968057 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784792.842422:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784792.842445:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 352 00000100:00100000:2.0:1787784792.842454:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658164352:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784792.842479:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29410:x1874627658163712:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 431us (1514us total) trans 4294968057 rc 0/0 00000100:00100000:2.0:1787784792.842479:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658164352:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (1306us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.842487:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1159 00000100:00100000:2.0:1787784792.842487:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 352 00000100:00080000:2.0:1787784792.843318:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784792.843324:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787784792.848781:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.848799:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.848808:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.848814:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784792.848819:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.848827:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.848846:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:2.0:1787784792.848849:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784792.848853:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.848860:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc66d0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.848927:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.850077:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658164480 00000100:00100000:1.0:1787784792.850101:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658164480 00000100:00100000:1.0:1787784792.850402:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1160 00000100:00100000:1.0:1787784792.850415:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29410:x1874627658164480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784792.850439:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784792.850458:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11e9 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.851008:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.851023:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.851044:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:1.0:1787784792.851047:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784792.851051:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784792.851249:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29410:x1874627658164480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 836us (2324us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.851258:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1160 00000100:00100000:1.0:1787784792.863082:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.863817:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658164608 00000100:00100000:2.0:1787784792.863828:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658164608 00000100:00100000:2.0:1787784792.863897:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:2.0:1787784792.863907:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658164608:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1787784792.864638:0:10688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:2.0:1787784792.864644:0:10688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb mtime from disk: 0 -> 1787784787 00002000:00010000:2.0:1787784792.864647:0:10688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb atime from disk: 0 -> 1787784787 00002000:00010000:2.0:1787784792.864649:0:10688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb ctime from disk: 0 -> 1787784787 00002000:00010000:2.0:1787784792.864651:0:10688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1787784792.864764:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658164608:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 856us (1689us total) trans 4294967368 rc 0/0 00000100:00100000:2.0:1787784792.864773:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:2.0:1787784792.875222:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.875351:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658164864 00000100:00100000:3.0:1787784792.875360:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658164864 00000100:00100000:3.0:1787784792.875405:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1161 00000100:00100000:3.0:1787784792.875414:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29410:x1874627658164864:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1787784792.875474:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.875490:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.875499:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 4/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.875529:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.875552:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 47 pid: 5812 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784792.875565:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cede00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668410880:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784792.875652:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f128780 going to sleep for 11 seconds 00000100:00100000:3.0:1787784792.877919:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cede00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668410880:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784792.877960:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.886624:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.886897:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658164992 00000100:00100000:1.0:1787784792.886906:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658164992 00010000:00010000:1.0:1787784792.886942:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 48 pid: 5812 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784792.887011:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 348 00000100:00100000:1.0:1787784792.887021:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2356:x1874627658164992:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784792.887037:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784792.887044:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 48 pid: 5812 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.887057:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 48 pid: 5812 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.887079:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6683 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11bf expref: 48 pid: 5812 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.887095:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:1.0:1787784792.887101:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.887111:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.887129:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.887200:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784792.887246:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658164992:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 226us (630us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.887253:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 348 00010000:00010000:2.0:1787784792.887772:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887792:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887824:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887831:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887837:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887843:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784792.887847:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887854:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887863:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:2.0:1787784792.887866:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787784792.887869:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 2/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.887875:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc66de lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.887928:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29410:x1874627658164864:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 12513us (12715us total) trans 0 rc -22/-22 00000100:00100000:2.0:1787784792.887936:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1161 00000100:00100000:2.0:1787784792.902787:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784792.902920:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658165120 00000100:00100000:1.0:1787784792.902940:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658165120 00000100:00100000:1.0:1787784792.903001:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 316 00000100:00100000:1.0:1787784792.903011:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658165120:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784792.903054:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.903075:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.903083:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784792.903087:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784792.903092:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.903135:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.903190:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 12 pid: 10688 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784792.903206:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc3c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668411008:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784792.903285:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ebd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784792.907999:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc3c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668411008:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784792.908032:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.910734:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784792.911018:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658165248 00000100:00100000:3.0:1787784792.911027:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658165248 00010000:00010000:3.0:1787784792.911065:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 13 pid: 10688 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784792.911107:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:3.0:1787784792.911127:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658165248:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784792.911143:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784792.911201:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 13 pid: 10688 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.911214:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 13 pid: 10688 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.911235:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc66c2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e2 expref: 13 pid: 10688 timeout: 683 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.911250:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784792.911253:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784792.911259:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.911269:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784792.911322:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784792.911371:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658165248:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 255us (645us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784792.911379:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 349 00010000:00010000:1.0:1787784792.911489:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911498:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911542:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911550:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911559:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911566:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784792.911570:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911577:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911589:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.911597:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc66e5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784792.911643:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658165120:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 8632us (8865us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784792.911651:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 316 00000100:00100000:1.0:1787784792.914592:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784792.914686:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658165504 00000100:00100000:0.0:1787784792.914695:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658165504 00000100:00100000:0.0:1787784792.914761:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1162 00000100:00100000:0.0:1787784792.914772:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29410:x1874627658165504:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784792.914842:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.914858:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.914870:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.915054:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.915065:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.915072:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.915103:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.915116:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 47 pid: 10850 timeout: 684 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.915200:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ade00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668411136:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784792.915270:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f900 going to sleep for 11 seconds 00000100:00100000:1.0:1787784792.918203:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ade00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668411136:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784792.918232:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784792.926014:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784792.926211:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658165632 00000100:00100000:2.0:1787784792.926221:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658165632 00010000:00010000:2.0:1787784792.926259:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 48 pid: 10850 timeout: 684 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784792.926299:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 350 00000100:00100000:2.0:1787784792.926309:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2357:x1874627658165632:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784792.926324:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784792.926331:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 48 pid: 10850 timeout: 684 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.926344:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 48 pid: 10850 timeout: 684 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.926367:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc66d7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11e9 expref: 48 pid: 10850 timeout: 684 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.926383:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:2.0:1787784792.926389:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.926398:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.926410:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784792.926443:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784792.926493:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658165632:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 185us (489us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784792.926501:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 350 00010000:00010000:0.0:1787784792.926639:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.926648:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784792.926826:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968058 00000004:02000000:0.0:1787784792.926958:0:5812:0:(mdd_object.c:3045:mdd_swap_layouts()) *** cfs_fail_loc=152, val=0*** 00000001:00080000:0.0:1787784792.935516:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968059 00010000:00010000:0.0:1787784792.939425:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939440:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939448:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939454:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784792.939458:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939465:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939480:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:0.0:1787784792.939483:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784792.939499:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939506:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc66f3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939627:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939635:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939641:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939647:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784792.939650:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939656:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939665:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:0.0:1787784792.939667:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784792.939686:0:5812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 94 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939698:0:5812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 94 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939709:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 94 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.939762:0:5812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 94 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.939779:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668411264:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784792.939926:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784792.943185:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668411264:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784792.943217:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943228:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6621 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943262:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943275:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943286:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943297:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943303:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943310:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943333:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784792.943337:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943344:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943356:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:0.0:1787784792.943359:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784792.943363:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943369:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc66fa lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943379:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943385:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943391:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943396:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784792.943399:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943404:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943423:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:0.0:1787784792.943426:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784792.943429:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 2/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.943434:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc66ec lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784792.943666:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29410:x1874627658165504:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 28894us (29080us total) trans 0 rc -95/-95 00000100:00100000:0.0:1787784792.943676:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1162 00000020:00080000:0.0:1787784792.943681:0:5812:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000040:00080000:2.0:1787784792.943885:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784792.943890:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784792.944426:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1787784792.944434:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784792.947280:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784792.947482:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658165888 00000100:00100000:0.0:1787784792.947492:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658165888 00000100:00100000:0.0:1787784792.947554:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 229 00000100:00100000:0.0:1787784792.947564:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29410:x1874627658165888:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1787784792.948379:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x21:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1787784792.948549:0:5814:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].17, 64 off9376 00000004:00080000:0.0:1787784792.948561:0:5814:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 584035413141ns, next at 585035410613ns, rc = 0 00000001:00080000:0.0:1787784792.948569:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968060 00010000:00010000:0.0:1787784792.948620:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.948636:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.948648:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.948668:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.948675:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.948683:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784792.948701:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787784792.948799:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29410:x1874627658165888:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1234us (1525us total) trans 4294968061 rc 0/0 00000100:00100000:0.0:1787784792.948808:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 229 00000020:00080000:0.0:1787784792.948814:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1787784792.948900:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784792.948908:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.948915:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784792.948919:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.948927:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.948941:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff989505542f00) 00010000:00010000:1.0:1787784792.948944:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787784792.948949:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.948955:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6701 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784792.948973:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784792.948975:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784792.948977:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784792.965268:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784792.965454:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658166016 00000100:00100000:0.0:1787784792.965463:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658166016 00000100:00100000:0.0:1787784792.965514:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:0.0:1787784792.965523:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ced680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29409:x1874627658166016:12345-192.168.203.43@tcp:49:diff.0 00000100:00100000:0.0:1787784792.965662:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ced680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29409:x1874627658166016:12345-192.168.203.43@tcp:49:diff.0 Request processed in 140us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784792.965678:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 230 00000100:00100000:2.0:1787784793.003190:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784793.003271:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658166272 00000100:00100000:0.0:1787784793.003281:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658166272 00000100:00100000:0.0:1787784793.003350:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 231 00000100:00100000:0.0:1787784793.003361:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29409:x1874627658166272:12345-192.168.203.43@tcp:35:diff.0 00000100:00100000:0.0:1787784793.003610:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:29409:x1874627658166272:12345-192.168.203.43@tcp:35:diff.0 Request processed in 250us (464us total) trans 4294968062 rc 0/0 00000100:00100000:0.0:1787784793.003634:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 231 00000100:00080000:2.0:1787784793.004289:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784793.009386:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784794.033370:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784794.289262:0:20689:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784794.289271:0:20689:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784794.289316:0:20689:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 3 00000040:00080000:1.0:1787784794.289328:0:20689:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784794.611088:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668411520:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611168:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784794.611218:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391283c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668412032:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611242:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784794.611268:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787784794.611273:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668411392:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611297:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784794.611313:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668411904:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611329:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784794.611353:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668411648:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611375:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784794.611390:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784794.611394:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539129a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668411776:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611414:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784794.611441:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668411520 00000100:00100000:2.0:1787784794.611447:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668411520 00000100:00100000:2.0:1787784794.611481:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668411904 00000100:00100000:2.0:1787784794.611484:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668411904 00000100:00100000:2.0:1787784794.611506:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668411648 00000100:00100000:2.0:1787784794.611509:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668411648 00000100:00100000:2.0:1787784794.611540:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:2.0:1787784794.611548:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668411520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611594:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668411520:12345-0@lo:400:kworker.0 Request processed in 47us (426us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784794.611600:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:2.0:1787784794.611616:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1787784794.611622:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668411904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611648:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668411904:12345-0@lo:400:kworker.0 Request processed in 26us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784794.611654:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1787784794.611664:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1787784794.611669:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035970c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668411648:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611708:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035970c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668411648:12345-0@lo:400:kworker.0 Request processed in 39us (332us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784794.611713:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1787784794.611730:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668411392 00000100:00100000:2.0:1787784794.611733:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668411392 00000100:00100000:2.0:1787784794.611752:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 564 00000100:00100000:2.0:1787784794.611757:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668411392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611783:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668411392:12345-0@lo:400:kworker.0 Request processed in 26us (486us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784794.611789:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 564 00000100:00080000:2.0:1787784794.611817:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784794.611864:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668411648:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611893:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668411392:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611902:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668411904:0@lo:400:kworker.0 00000100:00100000:2.0:1787784794.611910:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668411520:0@lo:400:kworker.0 00000004:00080000:1.0:1787784794.612330:0:8078:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 585699180361ns, next at 586699178014ns, rc = 0 00000100:00100000:1.0:1787784794.612353:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412032 00000100:00100000:1.0:1787784794.612357:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412032 00000100:00100000:1.0:1787784794.612394:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668411776 00000100:00100000:1.0:1787784794.612396:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668411776 00000100:00100000:1.0:1787784794.612422:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:1.0:1787784794.612430:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668412032:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784794.612475:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668412032:12345-0@lo:400:kworker.0 Request processed in 46us (1231us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784794.612481:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:1.0:1787784794.612493:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:1.0:1787784794.612499:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668411776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784794.612544:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668411776:12345-0@lo:400:kworker.0 Request processed in 46us (1129us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784794.612551:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00080000:1.0:1787784794.612582:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784794.612585:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784794.612605:0:3277:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967366 gen 2 00000004:00080000:1.0:1787784794.612610:0:3277:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989403597840, transno 4294967366 00000100:00100000:1.0:1787784794.612622:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391283c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668412032:0@lo:400:kworker.0 00000100:00100000:3.0:1787784794.613060:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784794.613072:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668412160:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784794.613108:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784794.613142:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412160 00000100:00100000:3.0:1787784794.613176:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412160 00000100:00100000:3.0:1787784794.613205:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:3.0:1787784794.613212:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ceed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668412160:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784794.613236:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784794.613274:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ceed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668412160:12345-0@lo:13:osp-pre-1-0.0 Request processed in 62us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784794.613279:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00100000:3.0:1787784794.613312:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668412160:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784794.614103:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539129a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668411776:0@lo:400:kworker.0 00010000:00010000:1.0:1787784795.057929:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784795.234066:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967356 is committed 00000100:00100000:1.0:1787784795.478343:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784795.479099:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658166528 00000100:00100000:2.0:1787784795.479109:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658166528 00000100:00100000:2.0:1787784795.479200:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1163 00000100:00100000:2.0:1787784795.479212:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:29432:x1874627658166528:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784795.479238:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784795.479258:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6708 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11f0 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784795.479372:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784795.479445:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.479460:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.479470:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.479486:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.479494:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.479501:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491200:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc671d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491221:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc671d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491247:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc671d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784795.491259:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:1.0:1787784795.491863:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491873:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491879:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491884:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784795.491888:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491895:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491906:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:1.0:1787784795.491909:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784795.491914:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491920:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc670f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491933:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.491948:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492817:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492828:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784795.492832:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492840:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492851:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff989520131c00) 00010000:00010000:1.0:1787784795.492854:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784795.492857:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492863:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6716 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492920:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6708 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11f0 expref: 99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492934:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc671d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11f0 expref: 98 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784795.492941:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:1.0:1787784795.492943:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784795.493132:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784795.493219:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:29432:x1874627658166528:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 14007us (14887us total) trans 4294968063 rc 0/0 00000100:00100000:1.0:1787784795.493227:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1163 00000100:00100000:1.0:1787784795.510425:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784795.513313:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658166656 00000100:00100000:2.0:1787784795.513327:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658166656 00000100:00100000:2.0:1787784795.513458:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1164 00000100:00100000:2.0:1787784795.513470:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:29432:x1874627658166656:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:2.0:1787784795.513548:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513567:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513579:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513604:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513612:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513619:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513625:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784795.513629:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513638:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513649:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:2.0:1787784795.513652:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784795.513657:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784795.513663:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc672b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784795.513752:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:29432:x1874627658166656:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 276us (3329us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784795.513760:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1164 00000100:00100000:2.0:1787784795.519872:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784795.520036:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658166912 00000100:00100000:0.0:1787784795.520045:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658166912 00000100:00100000:0.0:1787784795.520104:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 232 00000100:00100000:0.0:1787784795.520115:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:29432:x1874627658166912:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:0.0:1787784795.520681:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:29432:x1874627658166912:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 566us (822us total) trans 4294968064 rc 0/0 00000100:00100000:0.0:1787784795.520690:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 232 00010000:00010000:3.0:1787784796.087388:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784797.113364:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784797.681938:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668412288:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784797.682021:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784797.682068:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412288 00000100:00100000:1.0:1787784797.682075:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412288 00000100:00100000:1.0:1787784797.682135:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1787784797.682165:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668412288:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784797.682201:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784797.682253:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668412288:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784797.682259:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00080000:0.0:1787784797.682259:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784797.682288:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967356 gen 2 00000004:00080000:0.0:1787784797.682294:0:3279:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401cecb40, transno 4294967356 00000100:00100000:0.0:1787784797.682323:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668412288:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784797.960273:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784797.960372:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784797.960415:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784797.960448:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784797.960474:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784797.960507:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784797.960536:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784797.960563:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167296 00000100:00100000:1.0:1787784797.960572:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167296 00000100:00100000:1.0:1787784797.960617:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167808 00000100:00100000:1.0:1787784797.960620:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167808 00000100:00100000:1.0:1787784797.960630:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167424 00000100:00100000:1.0:1787784797.960632:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167424 00000100:00100000:1.0:1787784797.960640:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167680 00000100:00100000:1.0:1787784797.960643:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167680 00000100:00100000:1.0:1787784797.960677:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 355 00000100:00100000:1.0:1787784797.960686:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658167296:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784797.960760:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658167296:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 74us (387us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784797.960768:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 355 00000100:00100000:1.0:1787784797.960786:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 356 00000100:00100000:1.0:1787784797.960793:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658167808:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784797.960818:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658167808:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784797.960825:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 356 00000100:00100000:1.0:1787784797.960836:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 357 00000100:00100000:1.0:1787784797.960842:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658167424:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784797.960865:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658167424:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (368us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784797.960870:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 357 00000100:00100000:1.0:1787784797.960881:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 358 00000100:00100000:1.0:1787784797.960887:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658167680:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784797.960910:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658167680:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784797.960916:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 358 00000100:00100000:1.0:1787784797.960934:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167552 00000100:00100000:1.0:1787784797.960937:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167552 00000100:00100000:1.0:1787784797.960965:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167168 00000100:00100000:1.0:1787784797.960968:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167168 00000100:00100000:1.0:1787784797.960996:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:1.0:1787784797.961002:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2356:x1874627658167552:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784797.961039:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2356:x1874627658167552:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (624us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784797.961045:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:1.0:1787784797.961057:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:1.0:1787784797.961063:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:2356:x1874627658167168:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784797.961088:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:2356:x1874627658167168:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784797.961095:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00080000:0.0:1787784797.961573:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784797.961579:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784797.961581:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784797.961727:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167040 00000100:00100000:2.0:1787784797.961733:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167040 00000100:00100000:2.0:1787784797.961812:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 565 00000100:00100000:2.0:1787784797.961820:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658167040:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784797.961863:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658167040:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 44us (1600us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784797.961870:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 565 00000040:00080000:2.0:1787784798.065976:0:20837:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784798.065985:0:20837:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784798.066017:0:20837:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 3 00000040:00080000:2.0:1787784798.066023:0:20837:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787784798.130676:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784798.222165:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967368 is committed 00000001:00080000:1.0:1787784798.225082:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968064 is committed 00000001:00080000:1.0:1787784798.225098:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968061 is committed 00000004:00080000:1.0:1787784798.225102:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 18 committed 00000004:00080000:1.0:1787784798.225127:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 19 committed 00000020:00080000:1.0:1787784798.225134:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 23 batchid updated 00000020:00080000:3.0:1787784798.225180:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 23 00000020:00080000:3.0:1787784798.225192:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 23 00000100:00100000:3.0:1787784798.225511:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035943c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668412416:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1787784798.225577:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784798.225766:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412416 00000100:00100000:0.0:1787784798.225774:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412416 00000100:00100000:0.0:1787784798.225878:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1787784798.225889:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668412416:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1787784798.225916:0:6670:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:25 count 1 00002000:00010000:0.0:1787784798.225948:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x19:0x0] initial LVB size: 158, mtime: 0x6a8f6e53, atime: 0x6a8f6e53, ctime: 0x6a8f6e53, blocks: 0xb 00010000:00010000:0.0:1787784798.225956:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784798.225973:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=589s v=(0 0) 00010000:00010000:0.0:1787784798.225989:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00000100:00080000:2.0:1787784798.225997:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787784798.225997:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784798.226011:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784798.226022:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784798.226030:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784798.226052:0:6670:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787784798.226131:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784798.226138:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784798.226172:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784798.226176:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784798.226185:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784798.226202:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784798.226210:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6732 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784798.226226:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784798.226227:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784798.226229:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784798.226906:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668412416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1016us (1325us total) trans 4294967369 rc 0/0 00000100:00100000:0.0:1787784798.226915:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000004:00080000:3.0:1787784798.227331:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9894035943c0/2, rc 0, transno 4294967369 00000100:00100000:3.0:1787784798.227339:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035943c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668412416:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1787784799.155058:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784799.729520:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784799.729542:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668412544:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784799.729610:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784799.729677:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412544 00000100:00100000:1.0:1787784799.729684:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412544 00000100:00100000:1.0:1787784799.729728:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1787784799.729737:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668412544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784799.729779:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784799.729825:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668412544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784799.729842:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1787784799.729888:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668412544:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784799.732876:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668412800:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.732946:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784799.732992:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787784799.732999:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668412672:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733028:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784799.733047:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668413184:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733067:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784799.733091:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401ced2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668413056:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733116:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784799.733132:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784799.733137:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668412928:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733181:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784799.733207:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412800 00000100:00100000:3.0:1787784799.733213:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412800 00000100:00100000:3.0:1787784799.733253:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413184 00000100:00100000:3.0:1787784799.733256:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413184 00000100:00100000:3.0:1787784799.733278:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412928 00000100:00100000:3.0:1787784799.733280:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412928 00000100:00100000:3.0:1787784799.733296:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:3.0:1787784799.733303:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668412800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733351:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668412800:12345-0@lo:400:kworker.0 Request processed in 48us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784799.733358:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:3.0:1787784799.733374:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1787784799.733380:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ceda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668413184:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733408:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ceda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668413184:12345-0@lo:400:kworker.0 Request processed in 28us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784799.733413:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1787784799.733431:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1787784799.733436:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668412928:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733467:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668412928:12345-0@lo:400:kworker.0 Request processed in 31us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784799.733473:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00080000:3.0:1787784799.733505:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784799.733537:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668412928:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733566:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668413184:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.733574:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668412800:0@lo:400:kworker.0 00000100:00100000:1.0:1787784799.734059:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668412672 00000100:00100000:1.0:1787784799.734065:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668412672 00000100:00100000:1.0:1787784799.734105:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 566 00000100:00100000:1.0:1787784799.734113:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668412672:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784799.734171:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668412672:12345-0@lo:400:kworker.0 Request processed in 59us (1142us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784799.734179:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 566 00000100:00100000:1.0:1787784799.734210:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413056 00000100:00100000:1.0:1787784799.734214:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413056 00000100:00100000:1.0:1787784799.734243:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:1.0:1787784799.734250:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668413056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784799.734283:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668413056:12345-0@lo:400:kworker.0 Request processed in 33us (1166us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784799.734289:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00080000:3.0:1787784799.746728:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784799.746787:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401ced2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668413056:0@lo:400:kworker.0 00000100:00100000:3.0:1787784799.746846:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668412672:0@lo:400:kworker.0 00010000:00010000:1.0:1787784800.179493:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784801.201448:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784802.161507:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784802.161517:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784802.161795:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 2 00000040:00080000:2.0:1787784802.161802:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787784802.225360:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784802.801456:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668413312:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784802.801547:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784802.801755:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413312 00000100:00100000:1.0:1787784802.801765:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413312 00000100:00100000:1.0:1787784802.801840:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:1.0:1787784802.801850:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ced2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668413312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784802.801885:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784802.801932:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ced2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668413312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784802.801939:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:1.0:1787784802.801998:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668413312:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787784802.802490:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784803.079236:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784803.079271:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784803.079300:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784803.079330:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784803.079347:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784803.079366:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784803.079498:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168320 00000100:00100000:0.0:1787784803.079507:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168320 00000100:00100000:0.0:1787784803.079557:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168576 00000100:00100000:0.0:1787784803.079559:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168576 00000100:00100000:0.0:1787784803.079573:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168192 00000100:00100000:0.0:1787784803.079574:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168192 00000100:00100000:0.0:1787784803.079589:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 360 00000100:00100000:0.0:1787784803.079598:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658168320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784803.079668:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658168320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 71us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784803.079676:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 360 00000100:00100000:0.0:1787784803.079708:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 361 00000100:00100000:0.0:1787784803.079715:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658168576:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784803.079744:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658168576:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (443us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784803.079751:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 361 00000100:00100000:0.0:1787784803.079764:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 362 00000100:00100000:0.0:1787784803.079770:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658168192:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784803.079794:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658168192:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (446us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784803.079800:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 362 00000100:00080000:0.0:1787784803.079839:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784803.079842:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784803.080030:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168064 00000100:00100000:3.0:1787784803.080035:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168064 00000100:00100000:3.0:1787784803.080077:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168448 00000100:00100000:3.0:1787784803.080081:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168448 00000100:00100000:3.0:1787784803.080103:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:3.0:1787784803.080111:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2356:x1874627658168064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784803.080140:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2356:x1874627658168064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (914us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784803.080171:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:3.0:1787784803.080190:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:3.0:1787784803.080196:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2356:x1874627658168448:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784803.080222:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2356:x1874627658168448:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (856us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784803.080228:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:1.0:1787784803.080324:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784803.080345:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168704 00000100:00100000:1.0:1787784803.080349:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168704 00000100:00100000:1.0:1787784803.080375:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 363 00000100:00100000:1.0:1787784803.080382:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658168704:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784803.080481:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658168704:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 99us (156us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784803.080487:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 363 00000100:00100000:2.0:1787784803.080553:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658167936 00000100:00100000:2.0:1787784803.080557:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658167936 00000100:00100000:2.0:1787784803.080585:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 567 00000100:00100000:2.0:1787784803.080591:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658167936:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784803.080617:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658167936:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (1287us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784803.080623:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 567 00000100:00080000:0.0:1787784803.080933:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784803.080938:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784803.249548:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784803.360902:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967369 is committed 00010000:00010000:3.0:1787784804.274863:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784804.851091:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668413824:0@lo:400:kworker.0 00000100:00100000:3.0:1787784804.851180:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784804.851241:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784804.851247:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668413696:0@lo:400:kworker.0 00000100:00100000:3.0:1787784804.851274:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784804.851309:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413824 00000100:00100000:3.0:1787784804.851316:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413824 00000100:00100000:3.0:1787784804.851372:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:3.0:1787784804.851381:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668413824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784804.851432:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668413824:12345-0@lo:400:kworker.0 Request processed in 52us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784804.851440:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00080000:2.0:1787784804.851624:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784804.851638:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413696 00000100:00100000:2.0:1787784804.851642:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413696 00000100:00100000:2.0:1787784804.851677:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1787784804.851685:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668413696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784804.851722:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668413696:12345-0@lo:400:kworker.0 Request processed in 38us (446us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784804.851728:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00080000:2.0:1787784804.851744:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784804.851782:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668413696:0@lo:400:kworker.0 00000100:00100000:2.0:1787784804.851794:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668413824:0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.854882:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668413568:0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.854962:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784804.855029:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539374b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668414080:0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.855056:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784804.855086:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787784804.855094:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668413440:0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.855125:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784804.855189:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539376d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668413952:0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.855211:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784804.855244:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413568 00000100:00100000:0.0:1787784804.855251:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413568 00000100:00100000:0.0:1787784804.855305:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413952 00000100:00100000:0.0:1787784804.855309:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413952 00000100:00100000:0.0:1787784804.855337:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:0.0:1787784804.855345:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668413568:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.855398:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668413568:12345-0@lo:400:kworker.0 Request processed in 53us (436us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784804.855405:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:0.0:1787784804.855424:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1787784804.855430:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668413952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.855468:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668413952:12345-0@lo:400:kworker.0 Request processed in 39us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784804.855475:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1787784804.855518:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539376d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668413952:0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.855530:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668413568:0@lo:400:kworker.0 00000100:00100000:3.0:1787784804.855540:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668413440 00000100:00100000:3.0:1787784804.855544:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668413440 00000100:00100000:3.0:1787784804.855568:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 568 00000100:00100000:3.0:1787784804.855574:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668413440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784804.855621:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668413440:12345-0@lo:400:kworker.0 Request processed in 46us (495us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784804.855627:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 568 00000100:00100000:3.0:1787784804.855661:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414080 00000100:00100000:3.0:1787784804.855665:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414080 00000100:00100000:3.0:1787784804.855690:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:3.0:1787784804.855695:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668414080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784804.855725:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668414080:12345-0@lo:400:kworker.0 Request processed in 30us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784804.855731:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1787784804.855892:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668414208:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784804.855920:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784804.855950:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414208 00000100:00100000:1.0:1787784804.855953:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414208 00000100:00100000:1.0:1787784804.855977:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:1.0:1787784804.855982:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668414208:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784804.856013:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784804.856044:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668414208:12345-0@lo:13:osp-pre-1-0.0 Request processed in 62us (123us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784804.856050:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:1.0:1787784804.856079:0:3278:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967369 gen 2 00000004:00080000:1.0:1787784804.856084:0:3278:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9894035943c0, transno 4294967369 00000100:00100000:1.0:1787784804.856097:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668414208:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784804.856850:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668413440:0@lo:400:kworker.0 00000100:00100000:0.0:1787784804.856863:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539374b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668414080:0@lo:400:kworker.0 00000100:00080000:2.0:1787784804.857562:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784804.857569:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784805.299872:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784806.322256:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784807.345532:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784807.922592:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784807.922617:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668414336:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784807.922682:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784807.922808:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414336 00000100:00100000:1.0:1787784807.922816:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414336 00000100:00100000:1.0:1787784807.922882:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:1.0:1787784807.922891:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668414336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784807.922922:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00080000:2.0:1787784807.923109:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784807.923111:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668414336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 221us (429us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784807.923122:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:0.0:1787784807.925877:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668414336:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784808.202395:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784808.202450:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784808.202481:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784808.202513:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784808.202537:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784808.202554:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784808.202574:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784808.202600:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168832 00000100:00100000:1.0:1787784808.202608:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168832 00000100:00100000:1.0:1787784808.202650:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 569 00000100:00100000:1.0:1787784808.202658:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658168832:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784808.202718:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658168832:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784808.202725:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 569 00000100:00100000:1.0:1787784808.202874:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658169216 00000100:00100000:1.0:1787784808.202878:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658169216 00000100:00100000:1.0:1787784808.202911:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658169472 00000100:00100000:1.0:1787784808.202914:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658169472 00000100:00100000:1.0:1787784808.202925:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658169088 00000100:00100000:1.0:1787784808.202928:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658169088 00000100:00100000:1.0:1787784808.202940:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658169600 00000100:00100000:1.0:1787784808.202942:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658169600 00000100:00100000:1.0:1787784808.202956:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 366 00000100:00100000:1.0:1787784808.202961:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658169216:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784808.202990:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658168960 00000100:00100000:2.0:1787784808.202996:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658168960 00000100:00100000:2.0:1787784808.203025:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658169344 00000100:00100000:2.0:1787784808.203028:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658169344 00000100:00100000:2.0:1787784808.203050:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 410 00000100:00100000:2.0:1787784808.203057:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2355:x1874627658168960:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784808.203536:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658169216:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 572us (1083us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784808.203544:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 366 00000100:00100000:1.0:1787784808.203560:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 367 00000100:00100000:1.0:1787784808.203567:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658169472:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784808.203621:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 368 00000100:00100000:2.0:1787784808.203665:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2355:x1874627658168960:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 605us (1276us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784808.203674:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 410 00000100:00100000:2.0:1787784808.203690:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 411 00000100:00100000:2.0:1787784808.203697:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2355:x1874627658169344:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784808.203725:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2355:x1874627658169344:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (1170us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784808.203730:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 411 00000100:00100000:2.0:1787784808.203769:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 369 00000100:00100000:2.0:1787784808.203777:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658169600:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784808.203933:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658169600:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 157us (1357us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784808.203940:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 369 00000100:00080000:0.0:1787784808.204125:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784808.204133:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784808.204137:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784808.204722:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658169088:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784808.204765:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658169088:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1136us (2227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784808.204787:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 368 00000100:00100000:1.0:1787784808.212908:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658169472:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 9330us (10414us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784808.212921:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 367 00010000:00010000:1.0:1787784808.369335:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784809.395017:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784809.969621:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784809.969654:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668414464:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784809.969751:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784809.970012:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414464 00000100:00100000:1.0:1787784809.970023:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414464 00000100:00100000:1.0:1787784809.970109:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1787784809.970125:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668414464:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784809.970196:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784809.970263:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668414464:12345-0@lo:13:osp-pre-1-0.0 Request processed in 142us (511us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784809.970292:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1787784809.970349:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668414464:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1787784809.970596:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784809.971199:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035af0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668414848:0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.971259:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784809.971307:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784809.971313:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668414720:0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.971353:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784809.971390:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414848 00000100:00100000:0.0:1787784809.971396:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414848 00000100:00100000:0.0:1787784809.971460:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414720 00000100:00100000:0.0:1787784809.971464:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414720 00000100:00100000:0.0:1787784809.971487:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1787784809.971496:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668414848:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.971544:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668414848:12345-0@lo:400:kworker.0 Request processed in 50us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784809.971552:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1787784809.971577:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1787784809.971584:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668414720:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.971613:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668414720:12345-0@lo:400:kworker.0 Request processed in 29us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784809.971619:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00080000:0.0:1787784809.971647:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784809.971674:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668414720:0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.971684:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035af0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668414848:0@lo:400:kworker.0 00000100:00100000:2.0:1787784809.971770:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ad2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668414976:0@lo:400:kworker.0 00000100:00100000:2.0:1787784809.971809:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784809.971844:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784809.971850:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668414592:0@lo:400:kworker.0 00000100:00100000:2.0:1787784809.971893:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784809.971916:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035acb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668415104:0@lo:400:kworker.0 00000100:00100000:2.0:1787784809.971944:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784809.971973:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414592 00000100:00100000:2.0:1787784809.971977:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414592 00000100:00100000:2.0:1787784809.972005:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 570 00000100:00100000:2.0:1787784809.972012:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668414592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784809.972049:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668414592:12345-0@lo:400:kworker.0 Request processed in 38us (156us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784809.972056:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 570 00000100:00100000:2.0:1787784809.972104:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668414592:0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.972304:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668414976 00000100:00100000:0.0:1787784809.972309:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668414976 00000100:00100000:0.0:1787784809.972357:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:0.0:1787784809.972364:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668414976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.972404:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668414976:12345-0@lo:400:kworker.0 Request processed in 42us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784809.972410:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:0.0:1787784809.972441:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668415104 00000100:00100000:0.0:1787784809.972456:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668415104 00000100:00100000:0.0:1787784809.972485:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1787784809.972492:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668415104:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.972527:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668415104:12345-0@lo:400:kworker.0 Request processed in 36us (583us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784809.972533:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1787784809.972569:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035acb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668415104:0@lo:400:kworker.0 00000100:00100000:0.0:1787784809.972578:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ad2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668414976:0@lo:400:kworker.0 00000100:00080000:3.0:1787784809.972634:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784809.972639:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784810.074970:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.075173:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658169856 00000100:00100000:0.0:1787784810.075185:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658169856 00000100:00100000:0.0:1787784810.075276:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1165 00000100:00100000:0.0:1787784810.075290:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035961c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:29476:x1874627658169856:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:0.0:1787784810.075316:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784810.075341:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6739 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11f7 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.075489:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6740 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.075508:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6740 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.075526:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6740 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784810.075540:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1787784810.075757:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6739 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11f7 expref: 99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.075806:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6740 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11f7 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.075816:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:0.0:1787784810.075820:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784810.075824:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:0.0:1787784810.075905:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035961c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:29476:x1874627658169856:12345-192.168.203.43@tcp:101:diff.0 Request processed in 619us (959us total) trans 4294968065 rc 0/0 00000100:00100000:0.0:1787784810.075913:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1165 00000100:00080000:3.0:1787784810.077198:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784810.087510:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.087632:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658170112 00000100:00100000:0.0:1787784810.087646:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658170112 00000100:00100000:0.0:1787784810.087738:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 51 00000100:00100000:0.0:1787784810.087750:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:29476:x1874627658170112:12345-192.168.203.43@tcp:103:diff.0 00010000:00010000:0.0:1787784810.087771:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787784810.087806:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6740 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11f7 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.087835:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc671d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11f0 expref: 97 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.087849:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6628 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11aa expref: 96 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.087863:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9895198b5400) 00010000:00010000:0.0:1787784810.087875:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784810.087983:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29476:x1874627658170112:12345-192.168.203.43@tcp:103:diff.0 Request processed in 234us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.087993:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 51 00000100:00100000:1.0:1787784810.091483:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.091774:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658170240 00000100:00100000:0.0:1787784810.091789:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658170240 00000100:00100000:0.0:1787784810.091869:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1166 00000100:00100000:0.0:1787784810.091880:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29476:x1874627658170240:12345-192.168.203.43@tcp:58:diff.0 00010000:00010000:0.0:1787784810.092044:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.092065:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.092078:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784810.092546:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].33, 136 off12680 00000100:00100000:0.0:1787784810.093217:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29476:x1874627658170240:12345-192.168.203.43@tcp:58:diff.0 Request processed in 1330us (1741us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.093229:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1166 00000100:00100000:2.0:1787784810.108788:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.108886:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658170368 00000100:00100000:0.0:1787784810.108901:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658170368 00000100:00100000:0.0:1787784810.109003:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1167 00000100:00100000:0.0:1787784810.109016:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035943c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29476:x1874627658170368:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:0.0:1787784810.109050:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784810.109082:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b11fe expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.109211:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.109243:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11fe expref: 96 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.109266:0:20438:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.109282:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11fe expref: 96 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.109300:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784810.109303:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784810.109309:0:20438:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11fe expref: 96 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.109317:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2200, rc 0) 00000100:00100000:0.0:1787784810.109389:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035943c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:29476:x1874627658170368:12345-192.168.203.43@tcp:101:diff.0 Request processed in 376us (617us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.109397:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1167 00000040:00080000:3.0:1787784810.353823:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784810.353837:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784810.353890:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 34 count 3 00000040:00080000:3.0:1787784810.353898:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784810.354080:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895217592c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668415232:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784810.356898:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784810.363353:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784810.363584:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895217592c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668415232:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784810.364123:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658170496 00000100:00100000:0.0:1787784810.364139:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658170496 00000100:00080000:3.0:1787784810.364323:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784810.364634:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1168 00000100:00100000:0.0:1787784810.364649:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29478:x1874627658170496:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784810.364692:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784810.364831:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29478:x1874627658170496:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 186us (1491us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.364840:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1168 00000100:00100000:1.0:1787784810.368239:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.368397:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658170624 00000100:00100000:2.0:1787784810.368419:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658170624 00000100:00100000:2.0:1787784810.368502:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1169 00000100:00100000:2.0:1787784810.368513:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29478:x1874627658170624:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1787784810.368626:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368649:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368664:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368691:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368700:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368708:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368737:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368747:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368754:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784810.368803:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:2.0:1787784810.368808:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784810.368824:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368836:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368843:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368850:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368857:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784810.368862:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368871:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368885:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784810.368888:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784810.368894:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368900:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc675c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368917:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368923:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368930:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368935:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784810.368938:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.368944:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369223:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x329c2aeb (ffff989520131500) 00010000:00010000:2.0:1787784810.369228:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784810.369232:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369239:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6763 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369262:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369269:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369275:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369281:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784810.369284:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369290:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369298:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff98950a162200) 00010000:00010000:2.0:1787784810.369301:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784810.369304:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.369310:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc676a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784810.369419:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29478:x1874627658170624:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 898us (1185us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.369439:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1169 00000100:00100000:2.0:1787784810.376360:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.376831:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658170880 00000100:00100000:0.0:1787784810.376845:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658170880 00000100:00100000:0.0:1787784810.376922:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1170 00000100:00100000:0.0:1787784810.376934:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29478:x1874627658170880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784810.378284:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784810.378329:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6771 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1205 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.378487:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.378507:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.378521:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784810.378535:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1787784810.379553:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379572:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379580:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379597:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784810.379602:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379635:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379660:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784810.379664:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784810.379670:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379676:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6778 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379747:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6771 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1205 expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379760:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784810.379763:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784810.379767:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6771 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1205 expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.379777:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:0.0:1787784810.379880:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29478:x1874627658170880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2948us (3537us total) trans 4294968066 rc 301/301 00000100:00100000:0.0:1787784810.379891:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1170 00000100:00100000:3.0:1787784810.387275:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.387356:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658171008 00000100:00100000:0.0:1787784810.387369:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658171008 00000100:00100000:0.0:1787784810.387457:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1171 00000100:00100000:0.0:1787784810.387471:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29478:x1874627658171008:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784810.387559:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784810.387730:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29478:x1874627658171008:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 262us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.387739:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1171 00000100:00100000:1.0:1787784810.397802:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.403957:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658171264 00000100:00100000:2.0:1787784810.403981:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658171264 00000100:00100000:2.0:1787784810.404335:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1172 00000100:00100000:2.0:1787784810.404350:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:29478:x1874627658171264:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784810.404390:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784810.404430:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6786 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b120c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.405629:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc678d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.405652:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc678d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.405666:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc678d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784810.405679:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x22:0x0]: returning size 0 00010000:00010000:2.0:1787784810.405744:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6786 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b120c expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.405768:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc678d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b120c expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.405778:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff989520131500) 00010000:00010000:2.0:1787784810.405782:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784810.405787:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784810.405888:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29478:x1874627658171264:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1542us (8103us total) trans 4294968067 rc 0/0 00000100:00100000:2.0:1787784810.405897:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1172 00000100:00100000:3.0:1787784810.410162:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.410550:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658171520 00000100:00100000:0.0:1787784810.410563:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658171520 00000100:00100000:0.0:1787784810.410639:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1173 00000100:00100000:0.0:1787784810.410649:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29478:x1874627658171520:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784810.410724:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784810.410775:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc400/0xb0ce63137acc678d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b120c expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.410800:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff989520131500) 00010000:00010000:0.0:1787784810.410812:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784810.410843:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.410858:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.410869:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.410886:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.411516:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.411552:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412517:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412531:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412539:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412544:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784810.412549:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412557:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412572:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:0.0:1787784810.412575:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784810.412580:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412586:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc679b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412601:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412607:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412621:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412626:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784810.412629:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412636:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412644:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x2a052bbb (ffff98950a162b00) 00010000:00010000:0.0:1787784810.412646:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784810.412648:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.412654:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67a2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x2a052bbb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.412798:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29478:x1874627658171520:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 2146us (2664us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784810.412807:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1173 00010000:00010000:3.0:1787784810.417950:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784810.418503:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784810.418607:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658171648 00000100:00100000:3.0:1787784810.418620:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658171648 00000100:00100000:3.0:1787784810.418731:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1174 00000100:00100000:3.0:1787784810.418745:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29478:x1874627658171648:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784810.418823:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784810.418847:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc67a9 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1213 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.418886:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.418900:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.418908:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.419037:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc67a9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 48 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.419324:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 47 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.419348:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:3.0:1787784810.419351:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784810.419355:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784810.419528:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29478:x1874627658171648:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 786us (1034us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784810.419537:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1174 00000100:00100000:2.0:1787784810.427980:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.428274:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658171904 00000100:00100000:0.0:1787784810.428288:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658171904 00000100:00100000:0.0:1787784810.428381:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1175 00000100:00100000:0.0:1787784810.428395:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403597840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29478:x1874627658171904:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784810.428721:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.428747:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.428782:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784810.429356:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004e83174f reply state 00000000fb41f9e3 transno 4294968068 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.429368:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784810.429784:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403597840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29478:x1874627658171904:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 1386us (1812us total) trans 4294968068 rc 0/0 00000100:00100000:0.0:1787784810.429795:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1175 00000100:00100000:1.0:1787784810.441831:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:0.0:1787784810.442871:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.442901:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.442910:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.442916:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784810.442922:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.442930:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.442952:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:0.0:1787784810.442956:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784810.442961:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.443001:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc67b7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784810.443431:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658172032 00000100:00100000:2.0:1787784810.443446:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658172032 00000100:00100000:2.0:1787784810.443524:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1176 00000100:00100000:2.0:1787784810.443537:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:29478:x1874627658172032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784810.443569:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784810.443591:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 2/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b121a expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.443668:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.443684:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.443693:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784810.443697:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784810.443700:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784810.443797:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29478:x1874627658172032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 263us (1982us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.443805:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1176 00000100:00100000:2.0:1787784810.448039:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.453110:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658172160 00000100:00100000:0.0:1787784810.453132:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658172160 00000100:00100000:0.0:1787784810.453237:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 233 00000100:00100000:0.0:1787784810.453249:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d6450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29478:x1874627658172160:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:0.0:1787784810.453584:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d6450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29478:x1874627658172160:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 335us (5560us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.453593:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 233 00000100:00100000:3.0:1787784810.456708:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.458097:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658172416 00000100:00100000:0.0:1787784810.458116:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658172416 00000100:00100000:1.0:1787784810.458358:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 234 00000100:00100000:1.0:1787784810.458377:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:29478:x1874627658172416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784810.459138:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29478:x1874627658172416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 763us (2446us total) trans 4294968069 rc 0/0 00000100:00100000:1.0:1787784810.459223:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 234 00000100:00100000:2.0:1787784810.462119:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784810.462388:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658172672 00000100:00100000:3.0:1787784810.462400:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658172672 00000100:00100000:3.0:1787784810.462712:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1177 00000100:00100000:3.0:1787784810.462723:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29478:x1874627658172672:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784810.462867:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29478:x1874627658172672:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 145us (762us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784810.462875:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1177 00000100:00100000:2.0:1787784810.467382:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.467711:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658172928 00000100:00100000:0.0:1787784810.467724:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658172928 00000100:00100000:0.0:1787784810.467802:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1178 00000100:00100000:0.0:1787784810.467815:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29478:x1874627658172928:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784810.467845:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784810.467872:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc67c5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1221 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.468004:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.468022:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.468032:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 4/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.468096:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.468112:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 51 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.468129:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ac3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668415360:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784810.468284:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cf3000 going to sleep for 11 seconds 00000100:00100000:3.0:1787784810.481929:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ac3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668415360:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784810.481971:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784810.482586:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.489126:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658173056 00000100:00100000:0.0:1787784810.489207:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658173056 00010000:00010000:0.0:1787784810.502010:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 52 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.502082:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 351 00000100:00100000:0.0:1787784810.502096:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:2357:x1874627658173056:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784810.502116:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784810.502124:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 52 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.502140:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 52 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.502190:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc67be lrc: 0/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b121a expref: 52 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.502210:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:0.0:1787784810.502235:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.502246:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.502255:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.502287:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784810.502384:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2357:x1874627658173056:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 291us (19799us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.502393:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 351 00010000:00010000:3.0:1787784810.511066:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.511100:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784810.511132:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x22:0x0]: returning size 0 00000004:00080000:3.0:1787784810.511652:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x17:0x0], index 0: 0 00010000:00010000:3.0:1787784810.511868:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.511879:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.511887:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.511895:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784810.511900:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.511910:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.511931:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:3.0:1787784810.511935:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784810.511941:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 2/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.511949:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67cc lrc: 0/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.512026:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc67c5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1221 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.512039:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff989503a6da00) 00010000:00010000:3.0:1787784810.512042:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784810.512046:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc67c5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1221 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.512072:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:3.0:1787784810.512192:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29478:x1874627658172928:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 44378us (44836us total) trans 4294968070 rc 301/301 00000100:00100000:3.0:1787784810.512203:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1178 00000100:00100000:0.0:1787784810.521564:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.521714:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658173312 00000100:00100000:1.0:1787784810.521728:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658173312 00000100:00100000:1.0:1787784810.521807:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 235 00000100:00100000:1.0:1787784810.521831:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:29478:x1874627658173312:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784810.522379:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:29478:x1874627658173312:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 550us (826us total) trans 4294968071 rc 0/0 00000100:00100000:1.0:1787784810.522388:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 235 00000100:00100000:0.0:1787784810.528889:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.532214:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658173440 00000100:00100000:0.0:1787784810.532230:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658173440 00000100:00100000:0.0:1787784810.532294:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1179 00000100:00100000:0.0:1787784810.532318:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:29478:x1874627658173440:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784810.532348:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784810.532532:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc67da lrc: 2/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1228 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.532625:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67e1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.532642:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67e1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.532655:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67e1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784810.532710:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x22:0x0]: returning size 0 00000004:00200000:0.0:1787784810.532715:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784810.532736:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67e1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.532764:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc67da lrc: 0/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1228 expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.532781:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67e1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1228 expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.532790:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:0.0:1787784810.532792:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1787784810.532796:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2600, rc 0) 00000100:00100000:0.0:1787784810.532929:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29478:x1874627658173440:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 616us (4051us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.532937:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1179 00000100:00100000:1.0:1787784810.547499:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.548522:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658173568 00000100:00100000:2.0:1787784810.548538:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658173568 00000100:00080000:3.0:1787784810.548657:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784810.548963:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 318 00000100:00100000:2.0:1787784810.548980:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658173568:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784810.549006:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784810.549033:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b122f expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784810.549068:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x17:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784810.549095:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=601s v=(0 0) 00010000:00010000:2.0:1787784810.549100:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784810.549111:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.549133:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.549186:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784810.549269:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658173568:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 295us (1784us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.549277:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 318 00000100:00100000:1.0:1787784810.588103:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.594533:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658173824 00000100:00100000:2.0:1787784810.594564:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658173824 00000100:00100000:2.0:1787784810.594651:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1180 00000100:00100000:2.0:1787784810.594663:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29478:x1874627658173824:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784810.594839:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29478:x1874627658173824:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 178us (6754us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.594848:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1180 00000100:00100000:1.0:1787784810.606865:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.606915:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658173952 00000100:00100000:1.0:1787784810.606927:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658173952 00000100:00100000:1.0:1787784810.607010:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 319 00000100:00100000:1.0:1787784810.607024:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:29478:x1874627658173952:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784810.607063:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784810.607099:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1236 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.607172:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.607183:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784810.607529:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784810.607537:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 4/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.607587:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.607605:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 10 pid: 6670 timeout: 701 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.607623:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668415488:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784810.607748:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784810.625538:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.625596:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658174080 00000100:00100000:1.0:1787784810.625607:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658174080 00010000:00010000:1.0:1787784810.625680:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 11 pid: 6670 timeout: 701 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.625755:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 352 00000100:00100000:1.0:1787784810.625766:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658174080:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784810.625788:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784810.625818:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 11 pid: 6670 timeout: 701 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.625835:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 11 pid: 6670 timeout: 701 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.625870:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784810.625876:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784810.625886:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.625898:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.625953:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 11 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.625973:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbcb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668415616:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784810.626076:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5900 going to sleep for 11 seconds 00010000:00010000:1.0:1787784810.626277:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee5c6bd/0xb0ce63137acc67e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b122f expref: 11 pid: 6670 timeout: 701 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.626319:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668415488:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784810.626350:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.626362:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784810.626402:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:29478:x1874627658173952:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 19382us (19552us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784810.626423:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 319 00000100:00100000:1.0:1787784810.633924:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.633964:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658174208 00000100:00100000:1.0:1787784810.633975:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658174208 00002000:00100000:1.0:1787784810.634038:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x17:0x0] (0->157) req@ffff989404dbde00 x1874627658174208/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:691/0 lens 648/0 e 0 to 0 dl 1787784821 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784810.634051:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbde00 with x1874627658174208 ext(0->157) 00002000:00010000:1.0:1787784810.634057:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989404dbde00 00000100:00100000:1.0:1787784810.634087:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 103 00000100:00100000:1.0:1787784810.634097:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658174208:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1787784810.635922:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658174208:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 1811us (1997us total) trans 4294967357 rc 0/0 00000100:00100000:1.0:1787784810.635939:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 103 00002000:00100000:1.0:1787784810.635953:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x17:0x0] (0->157) req@ffff989404dbde00 x1874627658174208/t4294967357(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:691/0 lens 648/448 e 0 to 0 dl 1787784821 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784810.635977:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbde00 with x1874627658174208 ext(0->157) 00002000:00010000:1.0:1787784810.635984:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989404dbde00 00000100:00100000:1.0:1787784810.636215:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbcb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668415616:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784810.636244:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784810.636274:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658174080:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 10511us (10753us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784810.636284:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 352 00000100:00100000:1.0:1787784810.673379:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.673473:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658174336 00000100:00100000:1.0:1787784810.673485:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658174336 00000100:00100000:1.0:1787784810.673575:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1181 00000100:00100000:1.0:1787784810.673588:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29478:x1874627658174336:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1787784810.741641:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:29478:x1874627658174336:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 68042us (68264us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.741659:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1181 00000001:00080000:0.0:1787784810.741719:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968068 is committed 00000001:00080000:0.0:1787784810.741731:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968065 is committed 00000001:00080000:0.0:1787784810.741740:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968071 is committed 00000100:00100000:1.0:1787784810.744350:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.744610:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658174464 00000100:00100000:0.0:1787784810.744620:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658174464 00000100:00100000:0.0:1787784810.744669:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00100000:0.0:1787784810.744679:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658174464:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:3.0:1787784810.810396:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967357 is committed 00000100:00100000:0.0:1787784810.810854:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658174464:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 66167us (66503us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.810870:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00100000:1.0:1787784810.817697:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.817805:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658174720 00000100:00100000:1.0:1787784810.818541:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658174720 00000100:00100000:1.0:1787784810.818641:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1182 00000100:00100000:1.0:1787784810.818651:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29478:x1874627658174720:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784810.818735:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784810.818781:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc67e1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1228 expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.818810:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:1.0:1787784810.818816:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784810.819547:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.819564:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.819575:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784810.820660:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000096e26870 reply state 0000000042942fd2 transno 4294968072 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.820675:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784810.821675:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29478:x1874627658174720:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 3017us (3996us total) trans 4294968072 rc 0/0 00000100:00100000:1.0:1787784810.821690:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1182 00010000:00010000:2.0:1787784810.827253:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.827272:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.827280:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.827286:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784810.827291:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.827299:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.827317:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784810.827321:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784810.827326:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.827332:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc67f6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.827369:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.827937:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658174848 00000100:00100000:2.0:1787784810.827948:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658174848 00000100:00100000:2.0:1787784810.828011:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1183 00000100:00100000:2.0:1787784810.828020:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29478:x1874627658174848:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784810.828042:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784810.828060:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b123d expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.828121:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.828133:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.828195:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784810.828198:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784810.828201:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784810.828275:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29478:x1874627658174848:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 256us (916us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.828281:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1183 00000100:00100000:1.0:1787784810.843187:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.843226:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658174976 00000100:00100000:1.0:1787784810.843235:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658174976 00000100:00100000:1.0:1787784810.843985:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 321 00000100:00100000:1.0:1787784810.843999:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658174976:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1787784810.845731:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:1.0:1787784810.845740:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb mtime from disk: 0 -> 1787784787 00002000:00010000:1.0:1787784810.845743:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb atime from disk: 0 -> 1787784787 00002000:00010000:1.0:1787784810.845745:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb ctime from disk: 0 -> 1787784787 00002000:00010000:1.0:1787784810.845761:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1787784810.846640:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658174976:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 2639us (3506us total) trans 4294967358 rc 0/0 00000100:00100000:1.0:1787784810.846653:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 321 00000100:00100000:1.0:1787784810.850229:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784810.850538:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658175232 00000100:00100000:0.0:1787784810.850548:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658175232 00000100:00100000:0.0:1787784810.850600:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1184 00000100:00100000:0.0:1787784810.850611:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:29478:x1874627658175232:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1787784810.850682:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.850697:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 47 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.850707:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.850739:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 47 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.850754:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 47 pid: 20438 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.850766:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ae1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668415744:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784810.852004:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e840 going to sleep for 11 seconds 00000100:00100000:1.0:1787784810.854705:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ae1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668415744:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784810.854742:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.854957:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.860028:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658175360 00000100:00100000:1.0:1787784810.860051:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658175360 00010000:00010000:1.0:1787784810.860116:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 48 pid: 20438 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.860199:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 353 00000100:00100000:1.0:1787784810.860209:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2356:x1874627658175360:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784810.860236:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784810.860243:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 48 pid: 20438 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.860266:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 48 pid: 20438 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.860298:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc67b0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1213 expref: 48 pid: 20438 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.860325:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:1.0:1787784810.860330:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.860339:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.860389:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.860431:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784810.862013:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658175360:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1789us (7041us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784810.862035:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 353 00010000:00010000:1.0:1787784810.862217:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.862235:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.862276:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.862284:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.862293:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.862300:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784810.862305:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.862312:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.862326:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:1.0:1787784810.862337:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787784810.863576:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.863590:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6804 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.863661:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29478:x1874627658175232:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 13049us (13438us total) trans 0 rc -22/-22 00000100:00100000:1.0:1787784810.863671:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1184 00000100:00100000:0.0:1787784810.868220:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.868352:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658175488 00000100:00100000:1.0:1787784810.868362:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658175488 00000100:00100000:1.0:1787784810.868421:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 322 00000100:00100000:1.0:1787784810.868431:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658175488:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784810.868487:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.868508:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.868516:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784810.868521:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784810.868525:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 4/1,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.868557:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.868612:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 10 pid: 6671 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.868628:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668415872:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784810.868769:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a59c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784810.874665:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668415872:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784810.874705:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784810.880941:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.881734:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658175616 00000100:00100000:1.0:1787784810.881745:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658175616 00010000:00010000:2.0:1787784810.882063:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 11 pid: 6671 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784810.882227:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 354 00000100:00100000:2.0:1787784810.882238:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658175616:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784810.882294:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784810.882316:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 11 pid: 6671 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.882329:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 11 pid: 6671 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.882355:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc67ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1236 expref: 11 pid: 6671 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.882373:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784810.882377:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784810.882384:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.882395:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.882434:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784810.882548:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658175616:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 312us (1616us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.882556:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 354 00010000:00010000:0.0:1787784810.882648:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882659:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882695:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882704:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882722:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882729:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784810.882733:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882751:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882795:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.882812:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc680b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.882893:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658175488:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 14463us (14753us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.882912:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 322 00000100:00100000:3.0:1787784810.885340:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.886693:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658175872 00000100:00100000:1.0:1787784810.886720:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658175872 00000100:00100000:1.0:1787784810.886852:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1185 00000100:00100000:1.0:1787784810.886863:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:29478:x1874627658175872:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784810.886937:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.886969:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.887042:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.891031:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.891060:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.891070:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.891111:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784810.891127:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 47 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784810.891142:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668416000:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784810.893277:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063eb480 going to sleep for 11 seconds 00000100:00100000:1.0:1787784810.898040:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.898249:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668416000:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784810.898295:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.898344:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658176000 00000100:00100000:0.0:1787784810.898375:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658176000 00010000:00010000:0.0:1787784810.898450:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 48 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.899185:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 355 00000100:00100000:0.0:1787784810.899199:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2356:x1874627658176000:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784810.899229:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784810.899246:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 48 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.899269:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 48 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.899302:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc67fd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b123d expref: 48 pid: 10850 timeout: 701 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.899355:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:0.0:1787784810.899361:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.899382:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.899390:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.899438:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784810.899526:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.899551:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784810.899819:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968073 00000100:00100000:0.0:1787784810.899882:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2356:x1874627658176000:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 683us (1847us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.899893:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 355 00000001:00080000:2.0:1787784810.900032:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968074 00000001:00080000:2.0:1787784810.900135:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968075 00000100:00100000:2.0:1787784810.900279:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668416128:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787784810.900353:0:5813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784810.900393:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066776c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784810.900461:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668416128 00000100:00100000:0.0:1787784810.900491:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668416128 00000100:00100000:0.0:1787784810.900565:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:0.0:1787784810.900575:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668416128:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787784810.900739:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668416128:12345-0@lo:1000:mdt00_002.0 Request processed in 150us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784810.900749:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000100:00100000:0.0:1787784810.900810:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668416128:0@lo:1000:mdt00_002.0 00000001:00080000:0.0:1787784810.901485:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968076 00000100:00100000:0.0:1787784810.901588:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668416256:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787784810.901646:0:5813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784810.901673:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895113b50c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784810.901734:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668416256 00000100:00100000:2.0:1787784810.901746:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668416256 00000100:00100000:2.0:1787784810.901817:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1787784810.901827:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5813:x1874627668416256:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787784810.902331:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5813:x1874627668416256:12345-0@lo:1000:mdt00_002.0 Request processed in 503us (682us total) trans 4294967359 rc 0/0 00000100:00100000:2.0:1787784810.902341:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:1.0:1787784810.902538:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5813:1874627668416256:0@lo:1000:mdt00_002.0 00000001:00080000:1.0:1787784810.902807:0:5813:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968077 00010000:00010000:1.0:1787784810.902882:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.902892:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.902920:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.902926:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784810.902940:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.902948:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.902972:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:1.0:1787784810.902976:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784810.902981:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.902988:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6819 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903042:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903049:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903063:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903069:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784810.903072:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903098:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903108:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff98950a162200) 00010000:00010000:1.0:1787784810.903110:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784810.903127:0:5813:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11fe expref: 94 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903139:0:5813:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11fe expref: 94 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903314:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11fe expref: 94 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784810.903650:0:5813:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2200/0xb0ce63137acc6755 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b11fe expref: 94 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784810.903682:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbe940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668416384:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784810.903801:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784810.908293:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbe940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668416384:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784810.908340:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908352:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc674e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908398:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908429:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908442:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908455:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908462:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908469:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908475:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a3200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784810.908480:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908487:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908500:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784810.908521:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784810.908527:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908532:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc6820 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908555:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908561:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908566:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908570:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784810.908573:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908578:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908586:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff98950a162200) 00010000:00010000:0.0:1787784810.908589:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784810.908591:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.908597:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6812 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784810.908899:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:29478:x1874627658175872:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 22035us (23568us total) trans 4294968078 rc 0/0 00000100:00100000:0.0:1787784810.908921:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1185 00000020:00080000:0.0:1787784810.908928:0:5813:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:2.0:1787784810.909191:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784810.909197:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:3.0:1787784810.910781:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000040:00080000:2.0:1787784810.911419:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1787784810.911443:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784810.916953:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.917257:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658176128 00000100:00100000:2.0:1787784810.917273:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658176128 00000100:00100000:2.0:1787784810.917331:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658176384 00000100:00100000:2.0:1787784810.917334:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658176384 00000100:00100000:2.0:1787784810.917358:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 236 00000100:00100000:2.0:1787784810.917370:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29476:x1874627658176128:12345-192.168.203.43@tcp:49:diff.0 00000100:00100000:2.0:1787784810.917846:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29476:x1874627658176128:12345-192.168.203.43@tcp:49:diff.0 Request processed in 476us (7072us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.917855:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 236 00000100:00100000:2.0:1787784810.917879:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 237 00000100:00100000:2.0:1787784810.917887:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:29478:x1874627658176384:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1787784810.919502:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x22:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1787784810.919641:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968079 00010000:00010000:2.0:1787784810.919724:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.919747:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.919766:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.919776:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.919783:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.919790:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.919822:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787784810.919938:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:29478:x1874627658176384:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 2047us (3005us total) trans 4294968080 rc 0/0 00000100:00100000:2.0:1787784810.919965:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 237 00000020:00080000:2.0:1787784810.919972:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1787784810.920464:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784810.920474:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.920483:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784810.920486:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.920495:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.920510:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784810.920514:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787784810.920520:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.920527:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6827 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784810.920559:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784810.920561:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784810.920563:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784810.924965:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784810.925074:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658176512 00000100:00100000:2.0:1787784810.925087:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658176512 00000100:00100000:2.0:1787784810.925227:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 323 00000100:00100000:2.0:1787784810.925239:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:29476:x1874627658176512:12345-192.168.203.43@tcp:101:diff.0 00010000:00010000:2.0:1787784810.925264:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784810.925293:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc682e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1244 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784810.925338:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x17:0x0] initial LVB size: 158, mtime: 0x6a8f6e53, atime: 0x6a8f6e53, ctime: 0x6a8f6e53, blocks: 0xb 00010000:00010000:2.0:1787784810.925366:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=602s v=(0 0) 00010000:00010000:2.0:1787784810.925371:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784810.925383:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc682e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1244 expref: 16 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.925416:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f59f0f23/0xb0ce63137acc682e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1244 expref: 16 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784810.925433:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc400, rc 0) 00000100:00100000:2.0:1787784810.925509:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:29476:x1874627658176512:12345-192.168.203.43@tcp:101:diff.0 Request processed in 272us (575us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784810.925517:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 323 00000100:00080000:3.0:1787784810.925748:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784810.930284:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.930522:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658176640 00000100:00100000:1.0:1787784810.930540:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658176640 00002000:00100000:1.0:1787784810.930615:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x17:0x0] (0->4095) req@ffff98952175b480 x1874627658176640/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:691/0 lens 488/0 e 0 to 0 dl 1787784821 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784810.930628:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98952175b480 with x1874627658176640 ext(0->4095) 00002000:00010000:1.0:1787784810.930635:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98952175b480 00000100:00100000:2.0:1787784810.930819:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 104 00000100:00100000:2.0:1787784810.930841:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658176640:12345-192.168.203.43@tcp:3:diff.0 00000100:00100000:2.0:1787784810.931212:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658176640:12345-192.168.203.43@tcp:3:diff.0 Request processed in 378us (941us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784810.931220:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 104 00002000:00100000:2.0:1787784810.931229:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x17:0x0] (0->4095) req@ffff98952175b480 x1874627658176640/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:691/0 lens 488/4536 e 0 to 0 dl 1787784821 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784810.931238:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98952175b480 with x1874627658176640 ext(0->4095) 00002000:00010000:2.0:1787784810.931244:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98952175b480 00000100:00100000:2.0:1787784810.940527:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784810.940701:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658176896 00000100:00100000:1.0:1787784810.940744:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658176896 00000100:00100000:1.0:1787784810.940820:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 238 00000100:00100000:1.0:1787784810.940833:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:29476:x1874627658176896:12345-192.168.203.43@tcp:35:diff.0 00000100:00100000:1.0:1787784810.941228:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:29476:x1874627658176896:12345-192.168.203.43@tcp:35:diff.0 Request processed in 396us (714us total) trans 4294968081 rc 0/0 00000100:00100000:1.0:1787784810.941238:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 238 00010000:00010000:1.0:1787784811.449270:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784812.473160:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784812.822821:0:21015:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784812.822835:0:21015:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784812.823261:0:21015:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 34 count 3 00000040:00080000:1.0:1787784812.823276:0:21015:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:0.0:1787784812.983212:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784812.983230:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784812.983668:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 34 count 2 00000040:00080000:0.0:1787784812.983676:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784813.045303:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784813.045329:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401ced680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668416512:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784813.045409:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784813.047917:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668416512 00000100:00100000:0.0:1787784813.047931:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668416512 00000100:00100000:0.0:1787784813.048014:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:0.0:1787784813.048024:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668416512:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784813.048064:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85000192 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784813.048137:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668416512:12345-0@lo:13:osp-pre-0-0.0 Request processed in 115us (2728us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784813.048167:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00080000:2.0:1787784813.048254:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784813.048307:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401ced680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668416512:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784813.319304:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784813.319351:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784813.319406:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784813.319431:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784813.319454:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784813.319474:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784813.319496:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784813.319524:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177408 00000100:00100000:3.0:1787784813.319532:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177408 00000100:00100000:3.0:1787784813.319584:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177664 00000100:00100000:3.0:1787784813.319587:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177664 00000100:00100000:3.0:1787784813.319601:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177280 00000100:00100000:3.0:1787784813.319604:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177280 00000100:00100000:3.0:1787784813.319614:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177792 00000100:00100000:3.0:1787784813.319616:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177792 00000100:00100000:3.0:1787784813.319631:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 371 00000100:00100000:3.0:1787784813.319639:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658177408:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784813.319705:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658177408:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 66us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784813.319712:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 371 00000100:00100000:3.0:1787784813.319727:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 372 00000100:00100000:3.0:1787784813.319733:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658177664:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784813.319757:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658177664:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (326us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784813.319764:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 372 00000100:00100000:3.0:1787784813.319775:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 373 00000100:00100000:3.0:1787784813.319795:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658177280:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784813.319833:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658177280:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (377us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784813.319840:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 373 00000100:00100000:3.0:1787784813.319851:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:3.0:1787784813.319858:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658177792:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784813.319879:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658177792:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784813.319884:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00080000:3.0:1787784813.319906:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784813.319910:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784813.320100:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177152 00000100:00100000:0.0:1787784813.320106:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177152 00000100:00100000:0.0:1787784813.320142:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177536 00000100:00100000:0.0:1787784813.320173:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177536 00000100:00100000:0.0:1787784813.320197:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:0.0:1787784813.320205:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2354:x1874627658177152:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784813.320245:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:2354:x1874627658177152:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 41us (949us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784813.320252:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:0.0:1787784813.320270:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 416 00000100:00100000:0.0:1787784813.320276:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2355:x1874627658177536:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784813.320300:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2355:x1874627658177536:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (825us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784813.320306:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 416 00000100:00080000:2.0:1787784813.321261:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784813.321371:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177024 00000100:00100000:1.0:1787784813.321377:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177024 00000100:00100000:1.0:1787784813.321433:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 571 00000100:00100000:1.0:1787784813.321442:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658177024:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784813.321498:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658177024:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 57us (2091us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784813.321504:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 571 00010000:00010000:0.0:1787784813.489427:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784814.513372:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784814.819978:0:21066:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784814.819991:0:21066:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784814.820037:0:21066:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 34 count 2 00000040:00080000:3.0:1787784814.820043:0:21066:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784815.096808:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668416640:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784815.096904:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784815.096965:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668417152:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097015:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784815.097050:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784815.097056:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668417024:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097083:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784815.097120:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668416768:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097173:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784815.097196:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668417280:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097217:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784815.097235:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1787784815.097240:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668416896:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097277:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784815.097294:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668417408:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097314:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784815.097345:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668416640 00000100:00100000:0.0:1787784815.097354:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668416640 00000100:00100000:0.0:1787784815.097434:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1787784815.097445:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668416640:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784815.097489:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784815.097541:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668416640:12345-0@lo:13:osp-pre-1-0.0 Request processed in 98us (637us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784815.097549:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1787784815.097582:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417152 00000100:00100000:0.0:1787784815.097585:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417152 00000100:00100000:0.0:1787784815.097611:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417408 00000100:00100000:0.0:1787784815.097614:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417408 00000100:00100000:0.0:1787784815.097632:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:0.0:1787784815.097637:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668417152:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097678:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668417152:12345-0@lo:400:kworker.0 Request processed in 41us (663us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784815.097684:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00100000:0.0:1787784815.097696:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:0.0:1787784815.097702:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668417408:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097727:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668417408:12345-0@lo:400:kworker.0 Request processed in 26us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784815.097732:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:0.0:1787784815.097751:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417024 00000100:00100000:0.0:1787784815.097754:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417024 00000100:00100000:0.0:1787784815.097778:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417280 00000100:00100000:0.0:1787784815.097780:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417280 00000100:00100000:0.0:1787784815.097793:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668416896 00000100:00100000:0.0:1787784815.097795:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668416896 00000100:00100000:0.0:1787784815.097805:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:0.0:1787784815.097811:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668417024:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097838:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668417024:12345-0@lo:400:kworker.0 Request processed in 28us (754us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784815.097844:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:0.0:1787784815.097854:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1787784815.097858:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668417280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097878:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668417280:12345-0@lo:400:kworker.0 Request processed in 19us (660us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784815.097883:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1787784815.097891:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:0.0:1787784815.097896:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668416896:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.097926:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668416896:12345-0@lo:400:kworker.0 Request processed in 29us (648us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784815.097931:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00080000:0.0:1787784815.097958:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784815.097963:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784815.097965:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784815.098007:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668416896:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.098018:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668417408:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.098027:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668417280:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.098062:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668417024:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.098078:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668416640:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784815.098087:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668417152:0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.098190:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668416768 00000100:00100000:0.0:1787784815.098194:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668416768 00000100:00100000:0.0:1787784815.098217:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 572 00000100:00100000:0.0:1787784815.098223:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668416768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784815.101504:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668416768:12345-0@lo:400:kworker.0 Request processed in 3272us (4322us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784815.101519:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 572 00000100:00100000:0.0:1787784815.107788:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668416768:0@lo:400:kworker.0 00010000:00010000:0.0:1787784815.540227:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784816.284615:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968077 is committed 00000020:00080000:3.0:1787784816.284634:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98952482e840 refcount 2 committed 0 result 0 batchid 24 00000020:00080000:3.0:1787784816.284641:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98940513ee00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98952482e240 00000020:00080000:3.0:1787784816.284644:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98940513e000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895088c7000 00000001:00080000:3.0:1787784816.284659:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968081 is committed 00000001:00080000:3.0:1787784816.284665:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968078 is committed 00000001:00080000:3.0:1787784816.284671:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968080 is committed 00000020:00080000:3.0:1787784816.284699:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 24 committed batchid 23 00000020:00080000:3.0:1787784816.284933:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 23 00000001:00080000:3.0:1787784816.297306:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967358 is committed 00000001:00080000:3.0:1787784816.297333:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967359 is committed 00010000:00010000:0.0:1787784816.561815:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784817.586397:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784818.163317:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784818.163344:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668417536:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784818.163464:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784818.163550:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417536 00000100:00100000:0.0:1787784818.163558:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417536 00000100:00100000:0.0:1787784818.163622:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1787784818.163631:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668417536:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784818.163678:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85000192 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784818.163724:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668417536:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784818.163731:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00080000:0.0:1787784818.163761:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784818.163793:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967359 gen 2 00000004:00080000:0.0:1787784818.163799:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967359 committed transno 4294967359 00000100:00100000:0.0:1787784818.163831:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668417536:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784818.440292:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784818.440330:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784818.440370:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784818.440398:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784818.440418:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784818.440434:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784818.440459:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784818.440489:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178304 00000100:00100000:3.0:1787784818.440501:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178304 00000100:00100000:3.0:1787784818.440556:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178560 00000100:00100000:3.0:1787784818.440559:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178560 00000100:00100000:3.0:1787784818.440572:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178176 00000100:00100000:3.0:1787784818.440574:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178176 00000100:00100000:3.0:1787784818.440584:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178688 00000100:00100000:3.0:1787784818.440586:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178688 00000100:00100000:3.0:1787784818.440602:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 377 00000100:00100000:3.0:1787784818.440612:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658178304:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784818.440694:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2355:x1874627658178304:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 83us (363us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784818.440703:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 377 00000100:00100000:3.0:1787784818.440720:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 378 00000100:00100000:3.0:1787784818.440726:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658178560:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784818.440747:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658178560:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (377us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784818.440753:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 378 00000100:00100000:3.0:1787784818.440763:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 379 00000100:00100000:3.0:1787784818.440768:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658178176:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784818.440798:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658178176:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (379us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784818.440804:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 379 00000100:00100000:3.0:1787784818.440814:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 380 00000100:00100000:3.0:1787784818.440820:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658178688:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784818.440838:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658178688:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 18us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784818.440844:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 380 00000100:00100000:3.0:1787784818.440859:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658177920 00000100:00100000:3.0:1787784818.440862:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658177920 00000100:00100000:3.0:1787784818.440886:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:3.0:1787784818.440892:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658177920:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784818.440913:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658177920:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784818.440920:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:2.0:1787784818.441039:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178048 00000100:00100000:2.0:1787784818.441044:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178048 00000100:00100000:2.0:1787784818.441074:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178432 00000100:00100000:2.0:1787784818.441076:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178432 00000100:00100000:2.0:1787784818.441099:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:2.0:1787784818.441106:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2355:x1874627658178048:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784818.441135:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2355:x1874627658178048:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (858us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784818.441141:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:2.0:1787784818.441168:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 421 00000100:00100000:2.0:1787784818.441174:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2355:x1874627658178432:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784818.441194:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2355:x1874627658178432:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (759us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784818.441201:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 421 00000100:00080000:0.0:1787784818.441336:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784818.441342:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784818.441345:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787784818.611791:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784819.644644:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784820.209667:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668417664:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.209774:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784820.209834:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ad680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668418176:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.209864:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784820.209894:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784820.209900:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668418048:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.209927:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784820.209961:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668417792:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.209985:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784820.210016:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668418304:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.210039:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784820.210056:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784820.210061:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668417920:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.210083:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784820.210119:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417664 00000100:00100000:0.0:1787784820.210127:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417664 00000100:00100000:0.0:1787784820.210205:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:0.0:1787784820.210213:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668417664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.210266:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668417664:12345-0@lo:400:kworker.0 Request processed in 53us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784820.210274:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00000100:00100000:0.0:1787784820.210317:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668418176 00000100:00100000:0.0:1787784820.210320:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668418176 00000100:00100000:0.0:1787784820.212186:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417792 00000100:00100000:0.0:1787784820.212198:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417792 00000100:00100000:1.0:1787784820.212585:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784820.212602:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668418432:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784820.212667:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784820.212726:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668418432 00000100:00100000:1.0:1787784820.212730:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668418432 00000100:00100000:1.0:1787784820.212772:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1787784820.212782:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668418432:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784820.212838:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784820.212887:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668418432:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784820.212895:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1787784820.212948:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668418432:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784820.213759:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668418048 00000100:00100000:3.0:1787784820.213766:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668418048 00000100:00100000:3.0:1787784820.213821:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668418304 00000100:00100000:3.0:1787784820.213825:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668418304 00000100:00100000:3.0:1787784820.213849:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1787784820.213858:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668418048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784820.213915:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668418048:12345-0@lo:400:kworker.0 Request processed in 59us (3986us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784820.213921:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1787784820.213934:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1787784820.213949:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668418304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784820.213985:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668418304:12345-0@lo:400:kworker.0 Request processed in 38us (3946us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784820.213991:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1787784820.214226:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668417920 00000100:00100000:2.0:1787784820.214232:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668417920 00000100:00100000:2.0:1787784820.214266:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:2.0:1787784820.214273:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668418176:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784820.214315:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668418176:12345-0@lo:400:kworker.0 Request processed in 43us (4450us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784820.214321:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1787784820.214469:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668418304:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.214636:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:0.0:1787784820.214647:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668417792:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784820.214761:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:1.0:1787784820.214772:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668417920:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1787784820.216040:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784820.216049:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784820.216053:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784820.216188:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668417920:12345-0@lo:400:kworker.0 Request processed in 1414us (6099us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784820.216199:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:1.0:1787784820.216256:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668418048:0@lo:400:kworker.0 00000100:00100000:1.0:1787784820.216270:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668417664:0@lo:400:kworker.0 00000100:00100000:1.0:1787784820.216283:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ad680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668418176:0@lo:400:kworker.0 00000100:00100000:2.0:1787784820.216543:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668417920:0@lo:400:kworker.0 00000100:00100000:0.0:1787784820.216627:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668417792:12345-0@lo:400:kworker.0 Request processed in 1982us (6640us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784820.216635:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:2.0:1787784820.217618:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668417792:0@lo:400:kworker.0 00010000:00010000:0.0:1787784820.658031:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:2.0:1787784821.329304:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 24 batchid updated 00000020:00080000:2.0:1787784821.329324:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 24 00000020:00080000:2.0:1787784821.329343:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 24 00000100:00100000:0.0:1787784821.376020:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784821.376093:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178816 00000100:00100000:2.0:1787784821.376102:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178816 00000100:00100000:2.0:1787784821.376194:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1186 00000100:00100000:2.0:1787784821.376206:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29372:x1874627658178816:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1787784821.376309:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+44:29372:x1874627658178816:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 107us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784821.376317:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1186 00000100:00080000:3.0:1787784821.376949:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784821.681298:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784822.195890:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784822.195915:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784822.200704:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 1 00000040:00080000:0.0:1787784822.200719:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784822.705464:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784823.281430:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403124780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668418560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784823.281550:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784823.286036:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668418560 00000100:00100000:0.0:1787784823.286056:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668418560 00000100:00100000:0.0:1787784823.286203:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1787784823.286216:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668418560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784823.286268:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85000192 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784823.286391:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668418560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 178us (4841us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784823.286399:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1787784823.286822:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403124780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668418560:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787784823.295333:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784823.572598:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784823.572660:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784823.572695:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784823.572756:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784823.572788:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784823.572824:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179328 00000100:00100000:1.0:1787784823.572837:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179328 00000100:00100000:0.0:1787784823.572889:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179584 00000100:00100000:0.0:1787784823.572894:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179584 00000100:00100000:0.0:1787784823.572938:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179200 00000100:00100000:0.0:1787784823.572941:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179200 00000100:00080000:2.0:1787784823.573172:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784823.573178:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784823.575071:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179072 00000100:00100000:3.0:1787784823.575089:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179072 00000100:00100000:3.0:1787784823.575220:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 425 00000100:00100000:3.0:1787784823.575238:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ced680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2354:x1874627658179072:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784823.575362:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ced680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2354:x1874627658179072:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 131us (2789us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784823.575370:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 425 00000100:00100000:3.0:1787784823.575401:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658178944 00000100:00100000:3.0:1787784823.575405:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658178944 00000100:00100000:3.0:1787784823.575432:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 575 00000100:00100000:3.0:1787784823.575439:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658178944:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784823.575467:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658178944:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (2720us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784823.575474:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 575 00000100:00100000:3.0:1787784823.575554:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784823.575675:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784823.575729:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179712 00000100:00100000:3.0:1787784823.575734:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179712 00000100:00080000:2.0:1787784823.585331:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784823.585347:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784823.588100:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 383 00000100:00100000:1.0:1787784823.588224:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658179584:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784823.589312:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658179584:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1101us (16606us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784823.589339:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 383 00000100:00100000:1.0:1787784823.589389:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 384 00000100:00100000:1.0:1787784823.589403:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658179328:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784823.589495:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658179328:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 98us (16835us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784823.589502:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 384 00000100:00100000:1.0:1787784823.589548:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179456 00000100:00100000:1.0:1787784823.589558:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179456 00000100:00100000:1.0:1787784823.589635:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 426 00000100:00100000:1.0:1787784823.589643:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2354:x1874627658179456:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784823.589679:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2354:x1874627658179456:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (14126us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784823.589686:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 426 00000100:00080000:1.0:1787784823.589707:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784823.590021:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 385 00000100:00100000:0.0:1787784823.590035:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658179200:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784823.590071:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658179200:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 41us (17282us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784823.590078:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 385 00000100:00100000:3.0:1787784823.595174:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:3.0:1787784823.595200:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658179712:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784823.595315:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658179712:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 125us (19639us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784823.595338:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 386 00010000:00010000:0.0:1787784823.729429:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787784824.021489:0:21267:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 18:53:41 (1787784821) 00000100:00100000:2.0:1787784824.544502:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784824.544847:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179840 00000100:00100000:0.0:1787784824.544864:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179840 00000100:00100000:0.0:1787784824.546932:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1187 00000100:00100000:0.0:1787784824.546963:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:30097:x1874627658179840:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1787784824.547175:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:30097:x1874627658179840:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 199us (2659us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784824.547184:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1187 00000100:00100000:3.0:1787784824.577467:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784824.579050:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658179968 00000100:00100000:1.0:1787784824.579072:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658179968 00000100:00100000:2.0:1787784824.579359:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1188 00000100:00100000:2.0:1787784824.579373:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:30098:x1874627658179968:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784824.579399:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784824.579425:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6835 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b124b expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.579509:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.579526:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.579542:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.579558:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.579566:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.579573:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585232:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585258:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585267:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585276:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784824.585282:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585397:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585422:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:2.0:1787784824.585425:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.585431:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585437:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc683c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585591:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585598:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585604:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585609:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784824.585612:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585617:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585629:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff989520131b00) 00010000:00010000:2.0:1787784824.585632:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.585635:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585640:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6843 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585863:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6835 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b124b expref: 96 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585876:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:2.0:1787784824.585878:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.585882:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6835 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b124b expref: 96 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.585893:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784824.586072:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:30098:x1874627658179968:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6699us (8619us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784824.586084:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1188 00000100:00100000:2.0:1787784824.610876:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784824.611023:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658180224 00000100:00100000:0.0:1787784824.611036:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658180224 00000100:00100000:0.0:1787784824.611112:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1189 00000100:00100000:0.0:1787784824.611123:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:30098:x1874627658180224:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784824.611240:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784824.611274:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6461 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1109 expref: 95 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.611300:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:0.0:1787784824.611318:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784824.612187:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.612212:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.612223:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.612266:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 43 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.612284:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 43 pid: 5811 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784824.612337:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403124780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668418688:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784824.612452:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784824.622717:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784824.622790:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658180352 00000100:00100000:2.0:1787784824.622802:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658180352 00010000:00010000:2.0:1787784824.622863:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 44 pid: 5811 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784824.622924:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 356 00000100:00100000:2.0:1787784824.622941:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658180352:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784824.622963:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784824.622971:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 44 pid: 5811 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.622987:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 44 pid: 5811 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.623015:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:2.0:1787784824.623022:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.623032:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.623041:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.623064:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784824.628455:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc64a0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b111e expref: 44 pid: 5811 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784824.628809:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403124780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668418688:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784824.628846:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.628859:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.628865:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.628924:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.628960:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.628972:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784824.633697:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 512 00010000:00010000:1.0:1787784824.633721:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.633732:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.633740:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.633747:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784824.633763:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.633772:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.633792:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5f00) 00010000:00010000:1.0:1787784824.633795:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.633801:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.633807:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc684a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784824.633846:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c15b33d9 reply state 000000000f99050d transno 4294968082 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784824.633856:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784824.634975:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:2357:x1874627658180352:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 12029us (12261us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784824.634992:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 356 00000100:00100000:1.0:1787784824.641266:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:30098:x1874627658180224:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 30120us (30379us total) trans 4294968082 rc 0/0 00000100:00100000:1.0:1787784824.641288:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1189 00010000:00010000:0.0:1787784824.649641:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.649680:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.649689:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.649696:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784824.649701:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.649710:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.649732:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff98950a38c400) 00010000:00010000:0.0:1787784824.649736:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.649742:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.649749:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6851 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784824.675077:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784824.677403:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658180480 00000100:00100000:3.0:1787784824.677423:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658180480 00000100:00100000:3.0:1787784824.677817:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1190 00000100:00100000:3.0:1787784824.677829:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:30100:x1874627658180480:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784824.677863:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784824.677899:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6858 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1252 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.677995:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc685f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.678012:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc685f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.678026:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc685f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784824.678115:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784824.678120:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 30 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784824.678225:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc685f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.678285:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6858 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1252 expref: 97 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.678872:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc685f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1252 expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.678884:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:3.0:1787784824.678887:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784824.678899:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784824.678990:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:30100:x1874627658180480:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1163us (3928us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784824.679000:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1190 00000100:00100000:2.0:1787784824.686576:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784824.686888:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658180608 00000100:00100000:1.0:1787784824.686911:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658180608 00000100:00100000:1.0:1787784824.686995:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1191 00000100:00100000:1.0:1787784824.687007:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:30100:x1874627658180608:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784824.687037:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784824.687066:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6866 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1259 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687194:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687224:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687240:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687258:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687266:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687320:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687480:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc687b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687490:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc687b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687497:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc687b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784824.687548:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 512 00000004:00200000:1.0:1787784824.687552:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784824.687582:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc687b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687599:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687606:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687612:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687618:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784824.687623:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687630:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.687653:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:1.0:1787784824.687657:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.688112:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688124:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc686d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688144:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688191:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688397:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688406:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784824.688409:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688416:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688427:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff98950a38c300) 00010000:00010000:1.0:1787784824.688430:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.688434:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688440:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6874 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688808:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6866 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1259 expref: 98 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688828:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c200/0xb0ce63137acc687b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1259 expref: 97 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.688837:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:1.0:1787784824.688840:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784824.688842:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c200, rc 0) 00000100:00100000:1.0:1787784824.688938:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30100:x1874627658180608:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1932us (2376us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784824.688949:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1191 00000100:00100000:0.0:1787784824.696035:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784824.696866:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658180736 00000100:00100000:1.0:1787784824.696881:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658180736 00000100:00100000:1.0:1787784824.696979:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1192 00000100:00100000:1.0:1787784824.696991:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30100:x1874627658180736:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784824.697019:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784824.697044:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6882 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1260 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.697133:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.697171:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.697185:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.698407:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.698420:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.698430:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.698874:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.698886:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699070:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699079:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784824.699085:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699093:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699111:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:1.0:1787784824.699115:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.699121:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699128:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6889 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699219:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699227:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699233:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699239:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784824.699243:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699250:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699259:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff98950a38c300) 00010000:00010000:1.0:1787784824.699261:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.699265:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699271:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6890 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699417:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6882 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1260 expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699430:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:1.0:1787784824.699433:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.699444:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6882 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1260 expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.699472:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784824.700913:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30100:x1874627658180736:12345-192.168.203.43@tcp:101:bash.0 Request processed in 3905us (4876us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784824.700929:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1192 00000100:00100000:0.0:1787784824.716201:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784824.716348:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658180864 00000100:00100000:3.0:1787784824.716361:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658180864 00000100:00100000:3.0:1787784824.716440:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1193 00000100:00100000:3.0:1787784824.716461:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+43:30101:x1874627658180864:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784824.716496:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784824.716533:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6897 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1267 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.716624:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.716642:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.716656:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784824.716762:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784824.716767:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 30 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784824.716798:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.716830:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6897 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 45 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.716853:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.716863:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:3.0:1787784824.716867:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784824.716871:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784824.717992:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:30101:x1874627658180864:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1535us (1853us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784824.718005:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1193 00000100:00100000:3.0:1787784824.721407:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784824.721686:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658180992 00000100:00100000:0.0:1787784824.721698:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658180992 00000100:00100000:0.0:1787784824.721771:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1194 00000100:00100000:0.0:1787784824.721782:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+44:30101:x1874627658180992:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784824.721807:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784824.721835:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc68a5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b126e expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.721926:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.721944:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.721958:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.721975:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.721981:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.721988:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722135:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc68ba lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722176:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc68ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722208:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc68ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784824.722249:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 512 00000004:00200000:0.0:1787784824.722254:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784824.722277:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc68ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722292:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722298:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722306:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722323:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784824.722328:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722338:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722352:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:0.0:1787784824.722356:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.722361:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722369:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc68ac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722397:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722404:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722410:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722415:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784824.722418:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722424:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722431:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff98950a162d00) 00010000:00010000:0.0:1787784824.722434:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.722437:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722443:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc68b3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722495:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc68a5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b126e expref: 46 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.722510:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a3200/0xb0ce63137acc68ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b126e expref: 45 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.729115:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784824.729120:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784824.729125:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a3200, rc 0) 00000100:00100000:0.0:1787784824.729246:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30101:x1874627658180992:12345-192.168.203.43@tcp:101:rm.0 Request processed in 7459us (7847us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784824.729257:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1194 00000100:00100000:1.0:1787784824.734142:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784824.735014:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658181120 00000100:00100000:0.0:1787784824.735028:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658181120 00000100:00100000:0.0:1787784824.735098:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1195 00000100:00100000:0.0:1787784824.735110:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30101:x1874627658181120:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784824.735138:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784824.735188:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc68c1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1275 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735280:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735297:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735326:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735344:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735351:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735382:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735484:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735493:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735500:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.735507:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784824.735512:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.736961:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.736996:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784824.737000:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.737007:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737016:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68c8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737037:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737044:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737053:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737059:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500030600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784824.737063:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737071:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737079:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff98950a162b00) 00010000:00010000:0.0:1787784824.737082:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.737085:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737091:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68cf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737181:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc68c1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1275 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737194:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784824.737196:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.737200:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc68c1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1275 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.737213:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:0.0:1787784824.737327:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30101:x1874627658181120:12345-192.168.203.43@tcp:101:rm.0 Request processed in 2218us (3199us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784824.737341:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1195 00000100:00100000:3.0:1787784824.742761:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784824.743056:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658181248 00000100:00100000:2.0:1787784824.743069:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658181248 00000100:00100000:2.0:1787784824.743191:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1196 00000100:00100000:2.0:1787784824.743205:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30101:x1874627658181248:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784824.743235:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784824.743266:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc68d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b127c expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743375:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743394:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743408:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743424:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743432:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743439:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743546:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743555:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743562:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743569:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784824.743574:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743587:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743602:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:2.0:1787784824.743605:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.743611:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743618:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc68dd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743635:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743642:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743648:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743654:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784824.743657:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743675:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743682:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff989520131b00) 00010000:00010000:2.0:1787784824.743685:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.743688:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743698:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc68e4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743736:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc68d6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b127c expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743749:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:2.0:1787784824.743751:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.743755:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc68d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b127c expref: 46 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.743765:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784824.743847:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30101:x1874627658181248:12345-192.168.203.43@tcp:101:rm.0 Request processed in 645us (1100us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784824.743856:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1196 00010000:00010000:2.0:1787784824.753409:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784824.771637:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784824.774109:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658181376 00000100:00100000:0.0:1787784824.774131:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658181376 00000100:00100000:0.0:1787784824.774254:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1197 00000100:00100000:0.0:1787784824.774267:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30102:x1874627658181376:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784824.774299:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784824.774345:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68eb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1283 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774440:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774459:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774482:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774500:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774508:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774515:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774828:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774842:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774848:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774854:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784824.774859:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774866:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774881:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784824.774884:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.774890:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.774998:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc68f2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775017:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775024:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775029:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775034:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784824.775039:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775045:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775053:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff98950a162b00) 00010000:00010000:0.0:1787784824.775056:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.775059:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775065:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc68f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775110:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68eb lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1283 expref: 46 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775122:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784824.775125:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784824.775129:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc68eb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1283 expref: 46 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.775139:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:0.0:1787784824.775300:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30102:x1874627658181376:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1035us (3677us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784824.778063:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1197 00000100:00100000:0.0:1787784824.784953:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784824.785127:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658181504 00000100:00100000:1.0:1787784824.785141:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658181504 00000100:00100000:1.0:1787784824.785271:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1198 00000100:00100000:1.0:1787784824.785284:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30102:x1874627658181504:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784824.785325:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784824.785355:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6900 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b128a expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.785445:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.785641:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.785657:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.785965:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.785975:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786006:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786097:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786104:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786112:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786119:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784824.786123:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786133:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786173:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:1.0:1787784824.786177:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.786183:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786190:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6907 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786205:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786210:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786222:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786228:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784824.786231:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786237:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786245:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff98950a38c300) 00010000:00010000:1.0:1787784824.786248:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.786251:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.786256:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc690e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.787721:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6900 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b128a expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.787748:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:1.0:1787784824.787751:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784824.787761:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6900 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b128a expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.787777:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d800, rc 0) 00000100:00100000:1.0:1787784824.787888:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30102:x1874627658181504:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2606us (2950us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784824.787924:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1198 00000100:00100000:1.0:1787784824.797203:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784824.797402:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658181760 00000100:00100000:2.0:1787784824.797416:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658181760 00000100:00100000:2.0:1787784824.797509:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1199 00000100:00100000:2.0:1787784824.797920:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30102:x1874627658181760:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784824.797953:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784824.797983:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6915 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1291 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.798234:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.798259:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.798273:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.798290:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.798297:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.798317:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784824.802133:0:20438:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1a:0x0], index 1: 0 00010000:00010000:2.0:1787784824.802780:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802803:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802819:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802868:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802875:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802882:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802889:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784824.802906:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802915:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802932:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:2.0:1787784824.802936:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.802941:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.802948:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc691c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784824.802964:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a4188795 reply state 0000000088727565 transno 4294968083 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784824.802975:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784824.803026:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6915 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 48 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.803044:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 47 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.803051:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784824.803054:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784824.803056:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784824.803210:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30102:x1874627658181760:12345-192.168.203.43@tcp:101:cp.0 Request processed in 5689us (6021us total) trans 4294968083 rc 0/0 00000100:00100000:2.0:1787784824.803221:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1199 00010000:00010000:2.0:1787784824.809195:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.809230:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.809239:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.809245:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784824.809251:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.809260:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.809282:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff989520131b00) 00010000:00010000:2.0:1787784824.809286:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784824.809292:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.809300:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6923 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784824.813650:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784824.813870:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658181888 00000100:00100000:0.0:1787784824.813885:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658181888 00000100:00100000:0.0:1787784824.813974:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:0.0:1787784824.813988:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658181888:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784824.814013:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784824.814044:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1298 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784824.814082:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784824.814111:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=615s v=(0 0) 00010000:00010000:0.0:1787784824.814117:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784824.814129:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.814197:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.814216:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:0.0:1787784824.814293:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658181888:12345-192.168.203.43@tcp:101:cp.0 Request processed in 307us (657us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784824.814301:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:3.0:1787784824.821902:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784824.822005:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658182016 00000100:00100000:0.0:1787784824.822018:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658182016 00000100:00100000:0.0:1787784824.822091:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:0.0:1787784824.822104:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:30102:x1874627658182016:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784824.822129:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784824.822178:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b129f expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.822227:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.822239:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784824.822244:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784824.822250:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.822295:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.822323:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 12 pid: 10688 timeout: 715 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784824.822340:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668418816:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784824.822476:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784824.826492:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784824.826685:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658182144 00000100:00100000:2.0:1787784824.826699:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658182144 00010000:00010000:2.0:1787784824.826759:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 13 pid: 10688 timeout: 715 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784824.826819:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 357 00000100:00100000:2.0:1787784824.826832:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658182144:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784824.826854:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784824.826886:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 13 pid: 10688 timeout: 715 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.826911:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 13 pid: 10688 timeout: 715 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.826945:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784824.826950:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784824.826960:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.826972:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.827017:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784824.827035:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668418944:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784824.827185:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180840 going to sleep for 11 seconds 00010000:00010000:0.0:1787784824.829755:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000bfa7594/0xb0ce63137acc6938 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1298 expref: 13 pid: 10688 timeout: 715 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784824.829787:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668418816:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784824.829828:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784824.829851:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031000, rc 0) 00000100:00100000:0.0:1787784824.830125:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:30102:x1874627658182016:12345-192.168.203.43@tcp:101:cp.0 Request processed in 8021us (8245us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784824.830136:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:3.0:1787784824.839348:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784824.840670:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658182400 00000100:00100000:1.0:1787784824.840689:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658182400 00000100:00100000:1.0:1787784824.840779:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00100000:1.0:1787784824.840791:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d4450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30102:x1874627658182400:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784824.842044:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d4450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30102:x1874627658182400:12345-192.168.203.43@tcp:35:cp.0 Request processed in 1246us (2700us total) trans 4294968084 rc 0/0 00000100:00100000:1.0:1787784824.842058:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 239 00000100:00100000:2.0:1787784824.844049:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668418944:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784824.844095:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784824.844168:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658182144:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 17337us (17701us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784824.844181:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 357 00000100:00100000:1.0:1787784824.937250:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784824.937886:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658182528 00000100:00100000:3.0:1787784824.937909:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658182528 00000100:00100000:3.0:1787784824.937991:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1200 00000100:00100000:3.0:1787784824.938003:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30105:x1874627658182528:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784824.938031:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784824.938080:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6946 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12a6 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938214:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938232:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938249:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938267:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938274:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938281:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938871:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938884:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938904:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784824.938947:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:3.0:1787784824.938952:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784824.938973:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.938998:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939008:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939015:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939022:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939029:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784824.939034:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939043:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939058:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:3.0:1787784824.939062:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784824.939067:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939074:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc694d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939090:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939096:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939108:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939114:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784824.939117:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939124:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939132:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff9895198b5a00) 00010000:00010000:3.0:1787784824.939135:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784824.939137:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939143:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6954 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939238:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6946 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939255:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.939263:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:3.0:1787784824.939266:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784824.939268:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784824.939378:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30105:x1874627658182528:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1376us (2140us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784824.939397:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1200 00000100:00100000:0.0:1787784824.948886:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784824.949123:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658182656 00000100:00100000:1.0:1787784824.949458:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658182656 00000100:00100000:1.0:1787784824.949858:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 326 00000100:00100000:1.0:1787784824.949873:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658182656:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784824.949909:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784824.949998:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6962 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12ad expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784824.950081:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ad2c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668419072:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784824.952611:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503421b40 going to sleep for 11 seconds 00002000:00010000:3.0:1787784824.956861:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1787784824.956874:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb mtime: 0 -> 1787784824 00002000:00010000:3.0:1787784824.956878:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb atime: 0 -> 1787784824 00002000:00010000:3.0:1787784824.956881:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb ctime: 0 -> 1787784824 00000100:00100000:3.0:1787784824.957027:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ad2c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668419072:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784824.957081:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6962 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12ad expref: 13 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.957113:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6962 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12ad expref: 13 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784824.957128:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:3.0:1787784824.957250:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658182656:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7380us (8378us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784824.957260:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 326 00000100:00100000:0.0:1787784824.965577:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784824.965864:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658182784 00000100:00100000:1.0:1787784824.965878:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658182784 00000100:00100000:1.0:1787784824.965978:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1201 00000100:00100000:1.0:1787784824.965992:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30105:x1874627658182784:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784824.966044:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784824.966075:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6969 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12b4 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.966230:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.966247:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.966260:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784824.966351:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 512 00000004:00200000:1.0:1787784824.966356:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784824.966385:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.966417:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6969 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.966438:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784824.966449:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:1.0:1787784824.966452:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784824.966456:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c600, rc 0) 00000100:00100000:1.0:1787784824.966554:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30105:x1874627658182784:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 566us (993us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784824.966562:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1201 00000100:00100000:0.0:1787784824.971650:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784824.971937:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658182912 00000100:00100000:3.0:1787784824.971950:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658182912 00000100:00100000:3.0:1787784824.972020:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 327 00000100:00100000:3.0:1787784824.972031:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658182912:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784824.972054:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784824.972078:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ac822b5/0xb0ce63137acc6977 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12bb expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784824.972199:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668419200:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784824.972338:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989505495d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784824.976765:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668419200:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784824.976822:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ac822b5/0xb0ce63137acc6977 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12bb expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.976853:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ac822b5/0xb0ce63137acc6977 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12bb expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784824.976869:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:2.0:1787784824.977133:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658182912:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 5100us (5496us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784824.977189:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 327 00000100:00100000:0.0:1787784824.986511:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784824.986680:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658183168 00000100:00100000:2.0:1787784824.986693:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658183168 00000100:00100000:2.0:1787784824.986774:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1202 00000100:00100000:2.0:1787784824.986787:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30105:x1874627658183168:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:2.0:1787784824.987138:0:20438:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].34, 136 off12816 00000100:00100000:2.0:1787784824.987562:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30105:x1874627658183168:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 775us (1063us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784824.987572:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1202 00000040:00080000:2.0:1787784825.265691:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784825.265704:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784825.265751:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 35 count 2 00000040:00080000:2.0:1787784825.265760:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784825.266194:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668419328:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784825.266320:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784825.286370:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784825.286451:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658183296 00000100:00100000:3.0:1787784825.286472:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658183296 00000100:00100000:2.0:1787784825.287569:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668419328:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00080000:1.0:1787784825.287985:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784825.291503:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1203 00000100:00100000:3.0:1787784825.293063:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30115:x1874627658183296:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784825.293126:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784825.293322:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30115:x1874627658183296:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 1803us (6966us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784825.293343:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1203 00000100:00100000:3.0:1787784825.299978:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784825.300023:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658183552 00000100:00100000:3.0:1787784825.300033:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658183552 00000100:00100000:3.0:1787784825.302223:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1204 00000100:00100000:3.0:1787784825.302235:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30115:x1874627658183552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784825.302257:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784825.302279:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc697e lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12c2 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.303260:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6985 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.303279:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6985 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.303292:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6985 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784825.303637:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:3.0:1787784825.303789:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc697e lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c2 expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.303811:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6985 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c2 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.303820:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:3.0:1787784825.303823:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784825.303826:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784825.303911:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30115:x1874627658183552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1678us (3942us total) trans 4294968085 rc 0/0 00000100:00100000:3.0:1787784825.303920:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1204 00000100:00100000:3.0:1787784825.310878:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784825.310931:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784825.310962:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658183808 00000100:00100000:3.0:1787784825.310971:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658183808 00000100:00100000:3.0:1787784825.311031:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 328 00000100:00100000:3.0:1787784825.311052:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658183808:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784825.311105:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.311125:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 11 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.311135:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1a:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784825.311139:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784825.311144:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.311217:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 11 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.311231:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 11 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784825.311246:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668419456:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784825.311335:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895054956c0 going to sleep for 11 seconds 00000100:00080000:1.0:1787784825.312802:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784825.314944:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658183680 00000100:00100000:2.0:1787784825.314965:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658183680 00000100:00100000:2.0:1787784825.315087:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 52 00000100:00100000:2.0:1787784825.315099:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2357:x1874627658183680:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784825.315144:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784825.315196:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6985 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c2 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784825.315219:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784825.315226:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784825.315296:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658183680:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 199us (4427us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784825.315315:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 52 00000100:00100000:3.0:1787784825.316563:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989519d7b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668419456:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784825.316593:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784825.317691:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784825.317765:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658183936 00000100:00100000:1.0:1787784825.317781:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658183936 00002000:00100000:1.0:1787784825.317864:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1a:0x0] (0->157) req@ffff989519d7b480 x1874627658183936/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:706/0 lens 648/0 e 0 to 0 dl 1787784836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784825.317878:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989519d7b480 with x1874627658183936 ext(0->157) 00010000:00010000:1.0:1787784825.317888:0:18489:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 12 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.317907:0:18489:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 12 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784825.317917:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989519d7b480 00000100:00100000:1.0:1787784825.317956:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 13 00000100:00100000:1.0:1787784825.317968:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658183936:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784825.318976:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989519d7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658183936:12345-192.168.203.43@tcp:4:cp.0 Request processed in 1003us (1283us total) trans 4294967370 rc 0/0 00000100:00100000:1.0:1787784825.318989:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 13 00002000:00100000:1.0:1787784825.319000:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1a:0x0] (0->157) req@ffff989519d7b480 x1874627658183936/t4294967370(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:706/0 lens 648/448 e 0 to 0 dl 1787784836 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784825.319009:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989519d7b480 with x1874627658183936 ext(0->157) 00010000:00010000:1.0:1787784825.319017:0:18489:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 14 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.319035:0:18489:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 14 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784825.319052:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989519d7b480 00000100:00100000:0.0:1787784825.329436:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668419584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784825.329554:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784825.329635:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668419584 00000100:00100000:0.0:1787784825.329646:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668419584 00000100:00100000:0.0:1787784825.329725:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1787784825.329736:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668419584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784825.329763:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 1703936 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784825.329827:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668419584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784825.329835:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:3.0:1787784825.329929:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668419584:0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1787784825.330228:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784825.330246:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668419968:0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.330285:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784825.330322:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668419712:0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.330327:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784825.330332:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668420096:0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.330358:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784825.330361:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784825.330414:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668419840:0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.334259:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668419968 00000100:00100000:1.0:1787784825.334280:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668419968 00000100:00100000:1.0:1787784825.335192:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:1.0:1787784825.335206:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668419968:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.335346:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668419968:12345-0@lo:400:kworker.0 Request processed in 143us (5060us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.335355:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:1.0:1787784825.335394:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668420096 00000100:00100000:1.0:1787784825.335399:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668420096 00000100:00100000:1.0:1787784825.335429:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:1.0:1787784825.335436:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668420096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.335761:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668420096:12345-0@lo:400:kworker.0 Request processed in 324us (5400us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.335768:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00080000:1.0:1787784825.335790:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784825.335795:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784825.335838:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668420096:0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.335852:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668419968:0@lo:400:kworker.0 00000100:00100000:3.0:1787784825.335854:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668420224:0@lo:400:kworker.0 00000100:00100000:3.0:1787784825.335920:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784825.335966:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668420224 00000100:00100000:3.0:1787784825.335971:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668420224 00000100:00080000:1.0:1787784825.335999:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784825.336017:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1787784825.336023:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668420224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784825.336072:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668420224:12345-0@lo:400:kworker.0 Request processed in 48us (152us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784825.336085:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:0.0:1787784825.336107:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784825.336167:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668419840 00000100:00100000:3.0:1787784825.336171:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668419840 00000100:00100000:3.0:1787784825.336201:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1787784825.336209:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668419840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.336226:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668420224:0@lo:400:kworker.0 00000100:00100000:3.0:1787784825.336248:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668419840:12345-0@lo:400:kworker.0 Request processed in 40us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784825.336254:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1787784825.336280:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668419840:0@lo:400:kworker.0 00000100:00100000:2.0:1787784825.337170:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784825.337214:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668419712 00000100:00100000:2.0:1787784825.337221:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668419712 00000100:00100000:2.0:1787784825.337263:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 576 00000100:00100000:2.0:1787784825.337272:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668419712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784825.337278:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658184064 00000100:00100000:1.0:1787784825.337282:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658184064 00010000:00010000:1.0:1787784825.337322:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 14 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784825.337353:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668419712:12345-0@lo:400:kworker.0 Request processed in 82us (6990us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.337354:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 358 00000100:00100000:2.0:1787784825.337361:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 576 00000100:00100000:1.0:1787784825.337361:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658184064:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784825.337374:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784825.337403:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 14 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.337417:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 14 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.337448:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034a887f2/0xb0ce63137acc693f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b129f expref: 14 pid: 6669 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784825.337626:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989521758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668419712:0@lo:400:kworker.0 00010000:00010000:1.0:1787784825.337759:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784825.337764:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784825.337771:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.337783:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.337837:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784825.337887:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658184064:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 527us (724us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.337895:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 358 00010000:00010000:1.0:1787784825.338055:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338064:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338094:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338102:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338110:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338117:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784825.338121:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338130:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338186:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.338194:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e6b888ef/0xb0ce63137acc6993 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784825.338253:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658183808:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 27214us (27321us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.338262:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 328 00000100:00100000:2.0:1787784825.344256:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784825.344342:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658184320 00000100:00100000:3.0:1787784825.344358:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658184320 00000100:00100000:3.0:1787784825.344431:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1205 00000100:00100000:3.0:1787784825.344443:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:30115:x1874627658184320:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784825.344577:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:30115:x1874627658184320:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 135us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784825.344585:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1205 00000100:00100000:1.0:1787784825.358261:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784825.358479:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658184448 00000100:00100000:2.0:1787784825.358497:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658184448 00000100:00100000:2.0:1787784825.358590:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:2.0:1787784825.358603:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658184448:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784825.358627:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784825.358656:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12c9 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784825.358691:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1a:0x0] initial LVB size: 158, mtime: 0x6a8f6e78, atime: 0x0, ctime: 0x6a8f6e78, blocks: 0x1 00010000:00010000:2.0:1787784825.358718:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=616s v=(0 0) 00010000:00010000:2.0:1787784825.358723:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784825.358734:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784825.358755:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784825.358771:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784825.358867:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658184448:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 266us (616us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784825.358875:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:3.0:1787784825.379610:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784825.379736:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658184704 00000100:00100000:1.0:1787784825.379746:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658184704 00000100:00100000:1.0:1787784825.379816:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1206 00000100:00100000:1.0:1787784825.379829:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:30115:x1874627658184704:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784825.379954:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:30115:x1874627658184704:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 128us (356us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.379963:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1206 00000100:00100000:3.0:1787784825.399521:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784825.399705:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658184832 00000100:00100000:2.0:1787784825.399716:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658184832 00002000:00100000:2.0:1787784825.399767:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1a:0x0] (0->4095) req@ffff989402187c00 x1874627658184832/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:706/0 lens 488/0 e 0 to 0 dl 1787784836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784825.399785:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402187c00 with x1874627658184832 ext(0->4095) 00002000:00010000:2.0:1787784825.399791:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989402187c00 00000100:00100000:2.0:1787784825.399815:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 105 00000100:00100000:2.0:1787784825.399824:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658184832:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784825.400013:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658184832:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 190us (502us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784825.400020:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 105 00002000:00100000:2.0:1787784825.400026:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1a:0x0] (0->4095) req@ffff989402187c00 x1874627658184832/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:706/0 lens 488/4536 e 0 to 0 dl 1787784836 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784825.400033:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402187c00 with x1874627658184832 ext(0->4095) 00002000:00010000:2.0:1787784825.400037:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989402187c00 00000100:00100000:0.0:1787784825.426895:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784825.428384:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658184960 00000100:00100000:3.0:1787784825.428398:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658184960 00000100:00100000:3.0:1787784825.428450:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1207 00000100:00100000:3.0:1787784825.428460:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:30115:x1874627658184960:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784825.428480:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784825.428498:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69a1 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12d0 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.428535:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.428548:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.428560:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.428653:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69a1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.428669:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.428677:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:3.0:1787784825.428680:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784825.428683:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784825.428740:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30115:x1874627658184960:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 281us (1854us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784825.428747:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1207 00000100:00100000:0.0:1787784825.443098:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784825.443549:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658185088 00000100:00100000:1.0:1787784825.443559:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658185088 00000100:00100000:1.0:1787784825.443619:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1208 00000100:00100000:1.0:1787784825.443630:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30115:x1874627658185088:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784825.443687:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784825.443792:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30115:x1874627658185088:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 164us (704us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.443816:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1208 00000100:00100000:0.0:1787784825.449646:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784825.449768:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658185216 00000100:00100000:1.0:1787784825.449778:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658185216 00000100:00100000:1.0:1787784825.449828:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:1.0:1787784825.449839:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658185216:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784825.449891:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.449908:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784825.449912:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784825.449919:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.449933:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.449962:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.449970:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.449978:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.449985:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784825.450002:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.450011:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.450027:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.450034:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69af lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784825.450117:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658185216:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 280us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.450125:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:1.0:1787784825.453026:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784825.453143:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658185472 00000100:00100000:3.0:1787784825.453180:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658185472 00000100:00100000:3.0:1787784825.453229:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1209 00000100:00100000:3.0:1787784825.453238:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30115:x1874627658185472:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784825.453320:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.453473:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.453484:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.453519:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.453531:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 50 pid: 5812 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784825.453545:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402185680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668420352:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784825.453632:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895054956c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784825.458444:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784825.458568:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658185600 00000100:00100000:1.0:1787784825.458577:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658185600 00010000:00010000:1.0:1787784825.458620:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 51 pid: 5812 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784825.458658:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 359 00000100:00100000:1.0:1787784825.458667:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2357:x1874627658185600:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784825.458681:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784825.458686:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 51 pid: 5812 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.458711:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 51 pid: 5812 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.458733:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784825.458738:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.458746:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.458754:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784825.458774:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784825.458821:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2357:x1874627658185600:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 154us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784825.458828:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 359 00010000:00010000:3.0:1787784825.459070:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc69a8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d0 expref: 50 pid: 5812 timeout: 716 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784825.459086:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402185680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668420352:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784825.459185:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459195:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459200:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459404:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459413:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459419:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459425:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784825.459428:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459436:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459446:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:3.0:1787784825.459448:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784825.459451:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784825.459456:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc69b6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784825.459589:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30115:x1874627658185472:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 6351us (6570us total) trans 4294968086 rc 0/0 00000100:00100000:3.0:1787784825.459598:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1209 00000040:00080000:0.0:1787784825.459848:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784825.459852:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784825.460569:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:0.0:1787784825.460580:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784825.466242:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784825.466524:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658185856 00000100:00100000:1.0:1787784825.466533:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658185856 00000100:00100000:1.0:1787784825.466583:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 240 00000100:00100000:1.0:1787784825.466593:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30115:x1874627658185856:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784825.466832:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:30115:x1874627658185856:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 239us (597us total) trans 4294968087 rc 0/0 00000100:00100000:1.0:1787784825.466840:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 240 00010000:00010000:2.0:1787784825.783244:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784826.287060:0:21316:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784826.287069:0:21316:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784826.287108:0:21316:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 35 count 2 00000040:00080000:2.0:1787784826.287114:0:21316:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784826.395083:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784826.399594:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658186112 00000100:00100000:0.0:1787784826.399606:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658186112 00000100:00100000:0.0:1787784826.399681:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1210 00000100:00100000:0.0:1787784826.399692:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30121:x1874627658186112:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784826.399713:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784826.399733:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc69bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12d7 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.399837:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc69c4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.399854:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc69c4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.399867:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc69c4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784826.399879:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000100:00080000:1.0:1787784826.400692:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784826.402622:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc69bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d7 expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.402651:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc69c4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d7 expref: 101 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.402669:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:0.0:1787784826.402672:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784826.402676:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031000, rc 0) 00000100:00100000:0.0:1787784826.402745:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:30121:x1874627658186112:12345-192.168.203.43@tcp:101:cat.0 Request processed in 3054us (7670us total) trans 4294968088 rc 0/0 00000100:00100000:0.0:1787784826.402754:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1210 00000100:00100000:0.0:1787784826.411066:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784826.411670:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658186240 00000100:00100000:2.0:1787784826.411681:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658186240 00000100:00100000:2.0:1787784826.411743:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 53 00000100:00100000:2.0:1787784826.411754:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:2357:x1874627658186240:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784826.411770:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784826.411799:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc69c4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12d7 expref: 101 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.411822:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784826.411827:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784826.411899:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:2357:x1874627658186240:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 147us (839us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784826.411907:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 53 00000100:00100000:2.0:1787784826.414739:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784826.414806:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658186368 00000100:00100000:3.0:1787784826.414814:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658186368 00000100:00100000:3.0:1787784826.414893:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00100000:3.0:1787784826.414905:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2356:x1874627658186368:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784826.414927:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784826.414945:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12de expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.414970:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784826.414975:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784826.414984:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.415004:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.415018:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784826.415067:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658186368:12345-192.168.203.43@tcp:101:cat.0 Request processed in 165us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784826.415074:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00080000:2.0:1787784826.415264:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784826.425000:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784826.425087:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658186496 00000100:00100000:2.0:1787784826.425097:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658186496 00002000:00100000:2.0:1787784826.425201:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1a:0x0] (0->4095) req@ffff9894021861c0 x1874627658186496/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:707/0 lens 488/0 e 0 to 0 dl 1787784837 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784826.425214:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894021861c0 with x1874627658186496 ext(0->4095) 00002000:00010000:2.0:1787784826.425219:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894021861c0 00000100:00100000:2.0:1787784826.425284:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 106 00000100:00100000:2.0:1787784826.425354:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021861c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658186496:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:2.0:1787784826.425528:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021861c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658186496:12345-192.168.203.43@tcp:3:cat.0 Request processed in 235us (1687us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784826.425536:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 106 00002000:00100000:2.0:1787784826.425544:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1a:0x0] (0->4095) req@ffff9894021861c0 x1874627658186496/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:707/0 lens 488/4536 e 0 to 0 dl 1787784837 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784826.425553:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894021861c0 with x1874627658186496 ext(0->4095) 00002000:00010000:2.0:1787784826.425557:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894021861c0 00000100:00100000:0.0:1787784826.437656:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784826.440608:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658186752 00000100:00100000:1.0:1787784826.440623:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658186752 00000100:00100000:1.0:1787784826.440739:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 241 00000100:00100000:1.0:1787784826.440751:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30121:x1874627658186752:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:1.0:1787784826.441182:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:30121:x1874627658186752:12345-192.168.203.43@tcp:35:cat.0 Request processed in 432us (3534us total) trans 4294968089 rc 0/0 00000100:00100000:1.0:1787784826.441190:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 241 00000001:00080000:2.0:1787784826.445534:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968087 is committed 00000001:00080000:2.0:1787784826.445546:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968082 is committed 00000100:00100000:1.0:1787784826.485676:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784826.485803:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658187008 00000100:00100000:3.0:1787784826.485814:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658187008 00000100:00100000:1.0:1787784826.490002:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1211 00000100:00100000:1.0:1787784826.490024:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30122:x1874627658187008:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784826.490051:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784826.490072:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc69d9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12e5 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.490232:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc69e0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.490249:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc69e0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.490261:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc69e0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784826.490272:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:1.0:1787784826.490463:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc69d9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12e5 expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.490483:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc69e0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12e5 expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.490505:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784826.490507:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1787784826.490511:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784826.490636:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30122:x1874627658187008:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 621us (4968us total) trans 4294968090 rc 0/0 00000100:00100000:1.0:1787784826.490643:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1211 00000100:00100000:1.0:1787784826.508354:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784826.509576:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658187136 00000100:00100000:1.0:1787784826.509588:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658187136 00000100:00100000:1.0:1787784826.509645:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 332 00000100:00100000:1.0:1787784826.509655:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658187136:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:1.0:1787784826.509695:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.509715:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.509726:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.509735:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784826.509740:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784826.509744:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 5/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.509792:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.509807:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 13 pid: 6669 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784826.509822:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbfc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668420480:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784826.511068:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.511083:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 12 pid: 13825 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784826.511097:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668420608:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784826.511125:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405ddc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784826.515660:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784826.515682:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658187264 00000100:00100000:1.0:1787784826.515689:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658187264 00010000:00010000:1.0:1787784826.515724:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 14 pid: 6669 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784826.515760:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 360 00000100:00100000:1.0:1787784826.515769:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658187264:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784826.515795:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784826.515814:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 14 pid: 6669 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.515829:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 14 pid: 6669 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.516992:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784826.517724:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658187264:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1948us (2063us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784826.517736:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 360 00000100:00100000:1.0:1787784826.517809:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784826.520451:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc69d2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12de expref: 13 pid: 6669 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784826.520473:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbfc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668420480:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784826.520486:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668420608:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784826.520509:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784826.521208:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658187392 00000100:00100000:2.0:1787784826.521217:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658187392 00010000:00010000:2.0:1787784826.521251:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 13 pid: 13825 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784826.521287:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 361 00000100:00100000:2.0:1787784826.521294:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658187392:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784826.521320:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784826.521338:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 13 pid: 13825 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.521348:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 13 pid: 13825 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.521367:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc699a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12c9 expref: 13 pid: 13825 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.521378:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784826.521381:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784826.521385:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.521393:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.521417:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784826.521477:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658187392:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 182us (3666us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784826.521485:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 361 00010000:00010000:1.0:1787784826.523395:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523412:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523465:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523472:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523480:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523486:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784826.523491:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523500:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523517:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784826.523524:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e3ac749/0xb0ce63137acc69ee lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784826.523618:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2357:x1874627658187136:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 13962us (15270us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784826.523628:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 332 00000100:00100000:1.0:1787784826.529095:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784826.529544:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658187648 00000100:00100000:2.0:1787784826.529563:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658187648 00000100:00100000:2.0:1787784826.529619:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 242 00000100:00100000:2.0:1787784826.529630:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30122:x1874627658187648:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:2.0:1787784826.529707:0:5815:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x23:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc69e0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12e5 expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784826.531592:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968091 00000001:00080000:2.0:1787784826.532348:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968092 00010000:00010000:2.0:1787784826.532412:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.532431:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 45 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.532441:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 105 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.532449:0:5815:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.532490:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 105 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.532505:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 105 pid: 5813 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784826.532525:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912b840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668420736:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784826.532635:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 45 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.532644:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 45 pid: 20438 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784826.532653:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668420864:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784826.532675:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518ed6300 going to sleep for 11 seconds 00000100:00100000:2.0:1787784826.547424:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784826.548032:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658187776 00000100:00100000:0.0:1787784826.548042:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658187776 00010000:00010000:0.0:1787784826.548079:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 106 pid: 5813 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784826.548115:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 362 00000100:00100000:0.0:1787784826.548124:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658187776:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784826.548138:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784826.548169:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 106 pid: 5813 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.548185:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 106 pid: 5813 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.548208:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:0.0:1787784826.548213:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.548227:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.548233:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784826.548285:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658187776:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 162us (869us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784826.548310:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 362 00010000:00010000:3.0:1787784826.549491:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc695b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12a6 expref: 105 pid: 5813 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784826.549511:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912b840 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668420736:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784826.560165:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784826.560203:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658187904 00000100:00100000:3.0:1787784826.560211:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658187904 00010000:00010000:3.0:1787784826.560255:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 46 pid: 20438 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784826.560289:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 363 00000100:00100000:3.0:1787784826.560418:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658187904:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784826.560436:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784826.560444:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 46 pid: 20438 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.560459:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 46 pid: 20438 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.560481:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:3.0:1787784826.560487:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.560497:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.560505:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.560526:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784826.560575:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658187904:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 277us (437us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784826.560583:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 363 00010000:00010000:3.0:1787784826.560637:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc692a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1291 expref: 45 pid: 20438 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784826.560657:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668420864:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784826.560687:0:5815:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.560697:0:5815:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784826.560702:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784826.560828:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968093 00000001:00080000:3.0:1787784826.563269:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968094 00000001:00080000:3.0:1787784826.563356:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968095 00000100:00100000:3.0:1787784826.565219:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402185e00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668420992:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1787784826.565306:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784826.565348:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784826.565429:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668420992 00000100:00100000:2.0:1787784826.565436:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668420992 00000100:00100000:2.0:1787784826.565497:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:2.0:1787784826.565507:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668420992:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1787784826.565761:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668420992:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 254us (467us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784826.565768:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000100:00100000:2.0:1787784826.565810:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402185e00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668420992:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:2.0:1787784826.567061:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968096 00000100:00100000:2.0:1787784826.567132:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668421120:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1787784826.568269:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784826.569393:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7aee40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784826.569527:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668421120 00000100:00100000:0.0:1787784826.569536:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668421120 00000100:00100000:0.0:1787784826.569810:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:0.0:1787784826.569820:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668421120:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1787784826.570990:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668421120:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 1169us (2720us total) trans 4294967371 rc 0/0 00000100:00100000:0.0:1787784826.571001:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1787784826.571128:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668421120:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:2.0:1787784826.571181:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.571190:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.571197:0:5815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.571203:0:5815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784826.571208:0:5815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.571216:0:5815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.571231:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784826.571234:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1787784826.571239:0:5815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.571244:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69f5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784826.572658:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x49:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1787784826.572791:0:5815:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].18, 64 off9440 00000004:00080000:2.0:1787784826.572804:0:5815:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 617659656610ns, next at 618659653727ns, rc = 0 00000001:00080000:2.0:1787784826.572814:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968097 00010000:00010000:2.0:1787784826.573236:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784826.573241:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1787784826.573251:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc69e0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12e5 expref: 109 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784826.573743:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30122:x1874627658187648:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 44112us (44655us total) trans 4294968098 rc 0/0 00000100:00100000:2.0:1787784826.573754:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 242 00000020:00080000:2.0:1787784826.573760:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:3.0:1787784826.661485:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784826.663239:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658188160 00000100:00100000:1.0:1787784826.663253:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658188160 00000100:00100000:2.0:1787784826.663484:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1212 00000100:00100000:2.0:1787784826.663495:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30128:x1874627658188160:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784826.663518:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784826.663539:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69fc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12ec expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.664013:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.664030:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.664040:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.664054:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.664061:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.664068:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.664996:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a11 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665009:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a11 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665016:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a11 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784826.665028:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:2.0:1787784826.665120:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665128:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665134:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665140:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784826.665268:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665278:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665440:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:2.0:1787784826.665445:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784826.665450:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665461:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a03 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665476:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665482:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665489:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665494:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784826.665497:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665504:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665512:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff989520131800) 00010000:00010000:2.0:1787784826.665515:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784826.665518:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665524:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.665833:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc69fc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12ec expref: 111 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.668595:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a11 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12ec expref: 110 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.668604:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784826.668607:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784826.668610:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784826.668714:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:30128:x1874627658188160:12345-192.168.203.43@tcp:101:cat.0 Request processed in 5220us (7237us total) trans 4294968099 rc 0/0 00000100:00100000:2.0:1787784826.668724:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1212 00000100:00100000:2.0:1787784826.679954:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784826.680136:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658188416 00000100:00100000:0.0:1787784826.680184:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658188416 00000100:00100000:0.0:1787784826.680242:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 54 00000100:00100000:0.0:1787784826.680253:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:30128:x1874627658188416:12345-192.168.203.43@tcp:103:cat.0 00010000:00010000:0.0:1787784826.680270:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784826.680444:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a11 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12ec expref: 110 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.680465:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784826.680476:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784826.680530:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:30128:x1874627658188416:12345-192.168.203.43@tcp:103:cat.0 Request processed in 279us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784826.680542:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 54 00000100:00100000:3.0:1787784826.693614:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784826.693781:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658188544 00000100:00100000:0.0:1787784826.693791:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658188544 00000100:00100000:0.0:1787784826.693858:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1213 00000100:00100000:0.0:1787784826.693869:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:30128:x1874627658188544:12345-192.168.203.43@tcp:58:cat.0 00010000:00010000:0.0:1787784826.693974:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.693989:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784826.694001:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784826.694234:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].35, 136 off12952 00000100:00100000:0.0:1787784826.694516:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:30128:x1874627658188544:12345-192.168.203.43@tcp:58:cat.0 Request processed in 647us (909us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784826.694524:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1213 00000100:00100000:3.0:1787784826.698959:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784826.699088:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658188672 00000100:00100000:2.0:1787784826.699098:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658188672 00000100:00100000:2.0:1787784826.699187:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1214 00000100:00100000:2.0:1787784826.699196:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:30128:x1874627658188672:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784826.699214:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784826.699234:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12f3 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.699288:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.699312:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 110 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.699331:0:10850:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.699348:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 110 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.699360:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:2.0:1787784826.699363:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784826.699367:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 110 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784826.699375:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784826.699426:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:30128:x1874627658188672:12345-192.168.203.43@tcp:101:cat.0 Request processed in 231us (476us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784826.699433:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1214 00010000:00010000:0.0:1787784826.801533:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784827.507752:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784827.507762:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784827.507803:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 36 count 3 00000040:00080000:2.0:1787784827.507809:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784827.507931:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668421248:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784827.508028:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784827.511885:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668421248:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784827.513278:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.513763:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658188800 00000100:00100000:2.0:1787784827.513772:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658188800 00000100:00100000:2.0:1787784827.513847:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1215 00000100:00100000:2.0:1787784827.513860:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30129:x1874627658188800:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784827.513905:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784827.514005:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30129:x1874627658188800:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 150us (731us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784827.514013:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1215 00000100:00080000:1.0:1787784827.514088:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784827.517338:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.520262:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658188928 00000100:00100000:1.0:1787784827.520278:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658188928 00000100:00100000:1.0:1787784827.520361:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1216 00000100:00100000:1.0:1787784827.520373:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30129:x1874627658188928:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1787784827.521787:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521809:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521821:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521838:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521845:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521851:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521943:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521952:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.521959:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784827.522008:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:1.0:1787784827.522012:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784827.522025:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522036:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522042:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522048:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522053:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.522057:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522064:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522076:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784827.522078:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784827.522083:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522089:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6a2d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522102:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522118:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522124:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522129:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.522133:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522139:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522234:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x92f5931a (ffff98950a38cd00) 00010000:00010000:1.0:1787784827.522238:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784827.522241:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522259:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a34 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522274:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522280:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522302:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522308:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.522317:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522324:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522333:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:1.0:1787784827.522335:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784827.522338:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.522344:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a3b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.522428:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30129:x1874627658188928:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 2055us (5096us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.522437:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1216 00000100:00100000:3.0:1787784827.526421:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.526509:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658189184 00000100:00100000:2.0:1787784827.526519:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658189184 00000100:00100000:2.0:1787784827.526577:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1217 00000100:00100000:2.0:1787784827.526587:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30129:x1874627658189184:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784827.526607:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784827.526625:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a42 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b12fa expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526714:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526728:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526738:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784827.526748:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:2.0:1787784827.526834:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526841:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526847:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526852:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784827.526856:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526863:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526874:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:2.0:1787784827.526876:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784827.526881:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526886:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a49 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526923:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a42 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12fa expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526933:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784827.526935:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784827.526938:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a42 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12fa expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.526947:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784827.527003:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30129:x1874627658189184:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 418us (591us total) trans 4294968100 rc 301/301 00000100:00100000:2.0:1787784827.527010:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1217 00000100:00100000:3.0:1787784827.532409:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784827.532593:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658189312 00000100:00100000:0.0:1787784827.532603:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658189312 00000100:00100000:0.0:1787784827.532656:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1218 00000100:00100000:0.0:1787784827.532665:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30129:x1874627658189312:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784827.532700:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784827.532802:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30129:x1874627658189312:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 138us (402us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784827.532814:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1218 00000100:00100000:0.0:1787784827.549777:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.549866:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658189568 00000100:00100000:1.0:1787784827.549884:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658189568 00000100:00100000:1.0:1787784827.549940:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1219 00000100:00100000:1.0:1787784827.549950:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30129:x1874627658189568:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784827.549968:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784827.549986:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a57 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1301 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.550742:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a5e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.550757:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.550772:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784827.550783:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x24:0x0]: returning size 0 00010000:00010000:1.0:1787784827.550825:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a57 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1301 expref: 49 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.550841:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a5e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1301 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.550850:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784827.550852:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784827.550855:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784827.550921:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30129:x1874627658189568:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 972us (1152us total) trans 4294968101 rc 0/0 00000100:00100000:1.0:1787784827.550929:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1219 00000100:00100000:3.0:1787784827.561827:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.562113:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658189824 00000100:00100000:2.0:1787784827.562122:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658189824 00000100:00100000:2.0:1787784827.562202:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1220 00000100:00100000:2.0:1787784827.562212:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30129:x1874627658189824:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784827.562280:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784827.562314:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6a5e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1301 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.562331:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784827.562341:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784827.562361:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.562373:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 109 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.562382:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.562411:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 109 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.562424:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 109 pid: 5811 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784827.562438:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668421376:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784827.562515:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7aea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784827.566021:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.566117:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668421376:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784827.566143:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.566271:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658189952 00000100:00100000:1.0:1787784827.566278:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658189952 00010000:00010000:1.0:1787784827.566324:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 110 pid: 5811 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.566358:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 364 00000100:00100000:1.0:1787784827.566366:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:2356:x1874627658189952:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784827.566380:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784827.566387:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 110 pid: 5811 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.566404:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 110 pid: 5811 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.566426:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6970 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12b4 expref: 110 pid: 5811 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.566440:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:1.0:1787784827.566446:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.566455:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.566463:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.566487:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784827.566538:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:2356:x1874627658189952:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 172us (534us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.566545:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 364 00010000:00010000:3.0:1787784827.567450:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567461:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567485:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567496:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567504:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567731:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567739:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567746:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567751:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784827.567755:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567762:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567771:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:3.0:1787784827.567774:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784827.567778:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567784:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6a6c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567793:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567800:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567805:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567810:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784827.567813:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567818:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567825:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x3bec5d05 (ffff9895198b5a00) 00010000:00010000:3.0:1787784827.567826:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784827.567829:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.567834:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6a73 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x3bec5d05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.567940:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30129:x1874627658189824:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 5727us (6118us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784827.567947:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1220 00000100:00100000:3.0:1787784827.571441:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.571521:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658190080 00000100:00100000:2.0:1787784827.571531:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658190080 00000100:00100000:2.0:1787784827.571620:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1221 00000100:00100000:2.0:1787784827.571630:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30129:x1874627658190080:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784827.571649:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784827.571680:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a7a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1308 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.571714:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.571727:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.571737:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.571833:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6a7a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.571851:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.571859:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784827.571863:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784827.571867:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:2.0:1787784827.571928:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30129:x1874627658190080:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 299us (497us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784827.571935:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1221 00000100:00100000:3.0:1787784827.575687:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.575833:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658190336 00000100:00100000:1.0:1787784827.575841:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658190336 00000100:00100000:1.0:1787784827.575892:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1222 00000100:00100000:1.0:1787784827.575901:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30129:x1874627658190336:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784827.575970:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.575985:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.575997:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784827.576208:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cd44d00f reply state 00000000a85f5e46 transno 4294968102 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.576216:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784827.576310:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30129:x1874627658190336:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 408us (638us total) trans 4294968102 rc 0/0 00000100:00100000:1.0:1787784827.576317:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1222 00010000:00010000:1.0:1787784827.582938:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.582954:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.582962:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.582968:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.582973:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.582981:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.582998:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784827.583000:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787784827.583005:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.583024:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6a88 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.584138:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.584319:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658190464 00000100:00100000:2.0:1787784827.584329:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658190464 00000100:00100000:2.0:1787784827.584390:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1223 00000100:00100000:2.0:1787784827.584399:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:30129:x1874627658190464:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784827.584421:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784827.584438:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b130f expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.584490:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 50 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.584503:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 50 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.584512:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784827.584515:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784827.584518:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784827.584570:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30129:x1874627658190464:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 173us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784827.584578:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1223 00000100:00100000:0.0:1787784827.587341:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.587402:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658190592 00000100:00100000:1.0:1787784827.587410:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658190592 00000100:00100000:1.0:1787784827.587452:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 243 00000100:00100000:1.0:1787784827.587461:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30129:x1874627658190592:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1787784827.587571:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30129:x1874627658190592:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 110us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.587578:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 243 00000100:00100000:3.0:1787784827.594462:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.594967:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658190848 00000100:00100000:1.0:1787784827.594976:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658190848 00000100:00100000:1.0:1787784827.595124:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 244 00000100:00100000:1.0:1787784827.595135:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d5850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:30129:x1874627658190848:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784827.595397:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d5850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:30129:x1874627658190848:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 263us (944us total) trans 4294968103 rc 0/0 00000100:00100000:1.0:1787784827.595405:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 244 00000100:00100000:3.0:1787784827.599807:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784827.600077:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658191104 00000100:00100000:0.0:1787784827.600086:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658191104 00000100:00100000:0.0:1787784827.600173:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1224 00000100:00100000:0.0:1787784827.600183:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:30129:x1874627658191104:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784827.600270:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:30129:x1874627658191104:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 88us (471us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784827.600277:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1224 00000100:00100000:0.0:1787784827.603620:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.603790:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658191360 00000100:00100000:3.0:1787784827.603798:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658191360 00000100:00100000:3.0:1787784827.603853:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1225 00000100:00100000:3.0:1787784827.603862:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:30129:x1874627658191360:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784827.603888:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784827.603907:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6a96 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1316 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.603995:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.604009:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.604029:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 4/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.604061:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.604073:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 52 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.604086:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668421504:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784827.604388:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784827.611067:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.611181:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658191488 00000100:00100000:1.0:1787784827.611190:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658191488 00010000:00010000:1.0:1787784827.611226:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 53 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.611267:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 365 00000100:00100000:1.0:1787784827.611276:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2356:x1874627658191488:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784827.611303:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784827.611310:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 53 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.611322:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 53 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.611343:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784827.611349:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.611359:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.611367:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.611386:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784827.611435:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2356:x1874627658191488:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 159us (377us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.611443:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 365 00010000:00010000:0.0:1787784827.614361:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6a8f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b130f expref: 52 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784827.614385:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668421504:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784827.614408:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614418:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614423:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784827.614459:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x24:0x0]: returning size 0 00000004:00080000:0.0:1787784827.614777:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x18:0x0], index 0: 0 00010000:00010000:0.0:1787784827.614915:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614924:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614931:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614937:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784827.614942:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614948:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614974:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:0.0:1787784827.614977:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784827.614981:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.614987:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6a9d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.615369:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6a96 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1316 expref: 52 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.615380:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784827.615383:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784827.615387:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6a96 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1316 expref: 52 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.615397:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:0.0:1787784827.615574:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:30129:x1874627658191360:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 11711us (11961us total) trans 4294968104 rc 301/301 00000100:00100000:0.0:1787784827.615583:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1225 00000100:00100000:3.0:1787784827.621087:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.622283:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658191744 00000100:00100000:2.0:1787784827.622347:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658191744 00000100:00100000:2.0:1787784827.622431:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:2.0:1787784827.622441:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:30129:x1874627658191744:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784827.622774:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:30129:x1874627658191744:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 334us (1695us total) trans 4294968105 rc 0/0 00000100:00100000:2.0:1787784827.622782:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 245 00000100:00100000:3.0:1787784827.625104:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784827.625438:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658191872 00000100:00100000:0.0:1787784827.625448:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658191872 00000100:00100000:0.0:1787784827.625494:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1226 00000100:00100000:0.0:1787784827.625502:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:30129:x1874627658191872:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784827.625522:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784827.625541:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6aab lrc: 2/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b131d expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.625605:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6ab2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.625618:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6ab2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.625630:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6ab2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784827.625671:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x24:0x0]: returning size 0 00000004:00200000:0.0:1787784827.625676:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784827.625691:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6ab2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.625714:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031800/0xb0ce63137acc6aab lrc: 0/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b131d expref: 54 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.625731:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6ab2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b131d expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.625745:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:0.0:1787784827.625748:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1787784827.625752:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:0.0:1787784827.625799:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:30129:x1874627658191872:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 298us (704us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784827.625807:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1226 00000100:00100000:3.0:1787784827.629098:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.629263:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658192000 00000100:00100000:1.0:1787784827.629273:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658192000 00000100:00100000:1.0:1787784827.629501:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 333 00000100:00100000:1.0:1787784827.629511:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527dea050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658192000:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784827.629528:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784827.629547:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1324 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784827.629571:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784827.629590:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=618s v=(0 0) 00010000:00010000:1.0:1787784827.629825:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784827.629835:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 9 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.629853:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 9 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.629866:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784827.629947:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527dea050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658192000:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 438us (856us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.629954:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 333 00000100:00080000:2.0:1787784827.630234:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784827.635475:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.635578:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658192256 00000100:00100000:3.0:1787784827.635586:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658192256 00000100:00100000:3.0:1787784827.635637:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1227 00000100:00100000:3.0:1787784827.635653:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:30129:x1874627658192256:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784827.635752:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:30129:x1874627658192256:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 100us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784827.635760:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1227 00000100:00100000:0.0:1787784827.639342:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.639418:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658192384 00000100:00100000:2.0:1787784827.639427:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658192384 00000100:00100000:2.0:1787784827.639483:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 334 00000100:00100000:2.0:1787784827.639492:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:30129:x1874627658192384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784827.639508:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784827.639523:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b132b expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.639552:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.639561:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784827.639565:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784827.639571:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.639600:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.639613:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 10 pid: 6671 timeout: 718 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784827.639626:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668421632:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784827.639710:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7aea80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784827.642744:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.642903:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658192512 00000100:00100000:1.0:1787784827.642911:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658192512 00010000:00010000:1.0:1787784827.642946:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 11 pid: 6671 timeout: 718 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.642990:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 366 00000100:00100000:1.0:1787784827.642998:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658192512:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784827.643020:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784827.643040:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 11 pid: 6671 timeout: 718 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.643054:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 11 pid: 6671 timeout: 718 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.643083:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784827.643087:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784827.643095:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.643105:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.643135:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.643186:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668421760:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784827.643247:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895067c3780 going to sleep for 11 seconds 00010000:00010000:2.0:1787784827.643875:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6ab9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1324 expref: 11 pid: 6671 timeout: 718 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784827.643891:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668421632:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784827.643911:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.643922:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784827.643959:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:30129:x1874627658192384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 4469us (4627us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784827.643966:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 334 00000100:00100000:1.0:1787784827.647505:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668421760:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784827.647528:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784827.647578:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658192512:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 4580us (4840us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.647588:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 366 00000100:00100000:0.0:1787784827.651000:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.651179:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658192640 00000100:00100000:1.0:1787784827.651188:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658192640 00002000:00100000:1.0:1787784827.651235:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x18:0x0] (0->157) req@ffff9895392512c0 x1874627658192640/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:708/0 lens 648/0 e 0 to 0 dl 1787784838 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784827.651246:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895392512c0 with x1874627658192640 ext(0->157) 00002000:00010000:1.0:1787784827.651251:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9895392512c0 00000100:00100000:1.0:1787784827.651279:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 107 00000100:00100000:1.0:1787784827.651299:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658192640:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1787784827.651743:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392512c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658192640:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 455us (750us total) trans 4294967360 rc 0/0 00000100:00100000:1.0:1787784827.651751:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 107 00002000:00100000:1.0:1787784827.651758:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x18:0x0] (0->157) req@ffff9895392512c0 x1874627658192640/t4294967360(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:708/0 lens 648/448 e 0 to 0 dl 1787784838 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784827.651767:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895392512c0 with x1874627658192640 ext(0->157) 00002000:00010000:1.0:1787784827.651770:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9895392512c0 00000100:00100000:0.0:1787784827.656328:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.656402:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658192768 00000100:00100000:3.0:1787784827.656410:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658192768 00000100:00100000:3.0:1787784827.656461:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1228 00000100:00100000:3.0:1787784827.656471:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:30129:x1874627658192768:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1787784827.697969:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:30129:x1874627658192768:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 41489us (41638us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.697983:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1228 00000001:00080000:1.0:1787784827.698099:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968104 is committed 00000001:00080000:1.0:1787784827.698113:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968088 is committed 00000001:00080000:1.0:1787784827.698364:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968090 is committed 00000001:00080000:1.0:1787784827.698372:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968105 is committed 00000001:00080000:1.0:1787784827.698380:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968099 is committed 00000004:00080000:1.0:1787784827.698386:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 20 committed 00000100:00100000:2.0:1787784827.699478:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784827.699498:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668421888:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787784827.699558:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784827.699788:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668421888 00000100:00100000:3.0:1787784827.699795:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668421888 00000100:00100000:3.0:1787784827.699861:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:3.0:1787784827.699870:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668421888:12345-0@lo:6:osp-syn-1-0.0 00000100:00080000:2.0:1787784827.700029:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00080000:3.0:1787784827.700132:0:10688:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:26 count 1 00002000:00010000:3.0:1787784827.700188:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1a:0x0] initial LVB size: 158, mtime: 0x6a8f6e78, atime: 0x0, ctime: 0x6a8f6e78, blocks: 0x1 00010000:00010000:3.0:1787784827.700197:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.700214:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=618s v=(0 0) 00010000:00010000:3.0:1787784827.700218:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784827.700225:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.700243:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.700252:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.700261:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.700281:0:10688:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784827.700809:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668421888:12345-0@lo:6:osp-syn-1-0.0 Request processed in 938us (1249us total) trans 4294967372 rc 0/0 00000100:00100000:3.0:1787784827.700830:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000004:00080000:0.0:1787784827.701501:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff98953f7b2580/2, rc 0, transno 4294967372 00000100:00100000:0.0:1787784827.701509:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668421888:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1787784827.701577:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784827.701584:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.701594:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784827.701598:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.701608:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.701626:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.701636:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007447218f/0xb0ce63137acc6ac7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.701665:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784827.701667:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784827.701669:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:1.0:1787784827.705873:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9895012edb40 refcount 2 committed 0 result 0 batchid 25 00000020:00080000:1.0:1787784827.706078:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989505473c00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989518ed6300 00000020:00080000:1.0:1787784827.706082:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9894027cf000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9894035ed400 00000020:00080000:2.0:1787784827.706126:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 25 committed batchid 24 00000020:00080000:2.0:1787784827.706369:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 24 00000100:00100000:0.0:1787784827.707574:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.707788:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658192896 00000100:00100000:3.0:1787784827.707797:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658192896 00000100:00100000:3.0:1787784827.707891:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 336 00000100:00100000:3.0:1787784827.707902:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658192896:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1787784827.751262:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967360 is committed 00000100:00100000:1.0:1787784827.751472:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658192896:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 43566us (43898us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.751483:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 336 00000100:00100000:2.0:1787784827.756237:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784827.756413:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658193152 00000100:00100000:0.0:1787784827.756423:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658193152 00000100:00100000:0.0:1787784827.756515:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1229 00000100:00100000:0.0:1787784827.756525:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30129:x1874627658193152:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784827.756578:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784827.756605:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500030600/0xb0ce63137acc6ab2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b131d expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.756625:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:0.0:1787784827.756631:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784827.756768:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.756783:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.756792:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784827.757073:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009a255e62 reply state 00000000e58ef055 transno 4294968106 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784827.757082:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784827.757255:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30129:x1874627658193152:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 731us (1026us total) trans 4294968106 rc 0/0 00000100:00100000:0.0:1787784827.757263:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1229 00000100:00100000:3.0:1787784827.760473:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.761070:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658193280 00000100:00100000:1.0:1787784827.761081:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658193280 00000100:00100000:1.0:1787784827.761192:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1230 00000100:00100000:1.0:1787784827.761202:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30129:x1874627658193280:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784827.761227:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784827.761246:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1332 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.761320:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 49 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.761334:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 49 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.761355:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784827.761358:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784827.761361:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c600, rc 0) 00000100:00100000:1.0:1787784827.761424:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:30129:x1874627658193280:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 223us (958us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.761432:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1230 00010000:00010000:2.0:1787784827.761968:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.761978:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.761986:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.761992:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784827.761997:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.762004:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.762027:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:2.0:1787784827.762031:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784827.762036:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 2/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.762043:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6ace lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784827.765579:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.766115:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658193408 00000100:00100000:3.0:1787784827.766124:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658193408 00000100:00100000:3.0:1787784827.766208:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:3.0:1787784827.766219:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658193408:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1787784827.766464:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1787784827.766468:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 0 -> 1787784824 00002000:00010000:3.0:1787784827.766472:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb atime from disk: 0 -> 1787784825 00002000:00010000:3.0:1787784827.766474:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 0 -> 1787784824 00002000:00010000:3.0:1787784827.766476:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1787784827.766529:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658193408:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 312us (959us total) trans 4294967361 rc 0/0 00000100:00100000:3.0:1787784827.766537:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:3.0:1787784827.769495:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.769669:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658193664 00000100:00100000:1.0:1787784827.769679:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658193664 00000100:00100000:1.0:1787784827.769732:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1231 00000100:00100000:1.0:1787784827.769743:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30129:x1874627658193664:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1787784827.769799:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.769816:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.769826:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 4/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.769856:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.769869:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 48 pid: 10850 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.770032:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668422016:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784827.770109:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895034216c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784827.772829:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668422016:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784827.772857:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.774067:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.774196:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658193792 00000100:00100000:1.0:1787784827.774203:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658193792 00010000:00010000:1.0:1787784827.774237:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 49 pid: 10850 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.774267:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 367 00000100:00100000:1.0:1787784827.774275:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658193792:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784827.774300:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784827.774308:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 49 pid: 10850 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.774327:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 49 pid: 10850 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.774349:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6a81 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1308 expref: 49 pid: 10850 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.774363:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784827.774368:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.774377:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.774386:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.774417:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784827.774466:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2357:x1874627658193792:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 192us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.774474:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 367 00010000:00010000:3.0:1787784827.774923:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.774931:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.774960:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.774968:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.774975:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.774981:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1d800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784827.774985:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.774991:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.774999:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:3.0:1787784827.775002:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787784827.775006:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 2/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.775012:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6adc lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.775076:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30129:x1874627658193664:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 5334us (5588us total) trans 0 rc -22/-22 00000100:00100000:3.0:1787784827.775083:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1231 00000100:00100000:3.0:1787784827.779058:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.779139:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658193920 00000100:00100000:1.0:1787784827.779199:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658193920 00000100:00100000:1.0:1787784827.779254:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 338 00000100:00100000:1.0:1787784827.779265:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658193920:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784827.779319:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.779345:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.779355:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784827.779359:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784827.779364:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.779400:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.779413:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 10 pid: 13825 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.779426:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668422144:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784827.779492:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895034216c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784827.789851:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.789934:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658194048 00000100:00100000:3.0:1787784827.789943:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658194048 00010000:00010000:3.0:1787784827.789982:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 11 pid: 13825 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.790039:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 368 00000100:00100000:3.0:1787784827.790049:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658194048:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784827.790063:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784827.790083:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 11 pid: 13825 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.790096:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 11 pid: 13825 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.790121:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784827.790126:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784827.790132:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.790143:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.790196:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784827.790238:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658194048:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 191us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784827.790246:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 368 00010000:00010000:1.0:1787784827.791421:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000a39820/0xb0ce63137acc6ac0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b132b expref: 10 pid: 13825 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.791440:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668422144:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784827.791462:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791473:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791481:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791505:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791512:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791519:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791526:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784827.791531:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791539:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791553:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.791561:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6ae3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.791635:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658193920:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 12371us (12585us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.791706:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 338 00000100:00100000:0.0:1787784827.796531:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.796631:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658194304 00000100:00100000:2.0:1787784827.796648:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658194304 00000100:00100000:2.0:1787784827.796704:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1232 00000100:00100000:2.0:1787784827.796715:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30129:x1874627658194304:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784827.796789:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.796806:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.796816:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.797041:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.797053:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 48 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.797061:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.797092:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 48 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.797109:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 48 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784827.797123:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895402892c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668422272:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784827.802996:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784827.807038:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.807192:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658194432 00000100:00100000:1.0:1787784827.807201:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658194432 00010000:00010000:1.0:1787784827.807240:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 49 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.807280:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 369 00000100:00100000:1.0:1787784827.807303:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658194432:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784827.807320:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784827.807326:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 49 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.807340:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 49 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.807361:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784827.807366:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.807376:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.807384:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.807408:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784827.807459:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658194432:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 170us (429us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.807467:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 369 00010000:00010000:2.0:1787784827.808196:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6ad5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1332 expref: 48 pid: 20438 timeout: 718 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784827.808215:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895402892c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668422272:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784827.808235:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.808244:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.808250:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784827.808434:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968107 00000001:00080000:2.0:1787784827.808577:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968108 00000001:00080000:2.0:1787784827.808619:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968109 00000100:00100000:2.0:1787784827.810489:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895402892c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668422400:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787784827.810544:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784827.810621:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9c840 going to sleep for 11 seconds 00000100:00100000:3.0:1787784827.811360:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668422400 00000100:00100000:3.0:1787784827.811367:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668422400 00000100:00100000:3.0:1787784827.811464:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 55 00000100:00100000:3.0:1787784827.811474:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668422400:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787784827.811563:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668422400:12345-0@lo:1000:mdt00_003.0 Request processed in 91us (1018us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784827.811569:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 55 00000100:00100000:1.0:1787784827.811700:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895402892c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668422400:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1787784827.812276:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968110 00000100:00100000:1.0:1787784827.812470:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668422528:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787784827.812507:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784827.812581:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00080000:2.0:1787784827.812846:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784827.812956:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668422528 00000100:00100000:3.0:1787784827.812961:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668422528 00000100:00100000:3.0:1787784827.813038:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:3.0:1787784827.813046:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668422528:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787784827.813249:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:10850:x1874627668422528:12345-0@lo:1000:mdt00_003.0 Request processed in 203us (741us total) trans 4294967362 rc 0/0 00000100:00100000:3.0:1787784827.813257:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:1.0:1787784827.813341:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668422528:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1787784827.813415:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968111 00010000:00010000:1.0:1787784827.813442:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813452:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813460:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813466:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fc600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.813471:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813478:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813492:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff989520131800) 00010000:00010000:1.0:1787784827.813496:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784827.813500:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813505:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6af1 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813534:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813540:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813546:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813551:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989500031000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.813554:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813560:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813568:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:1.0:1787784827.813570:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784827.813577:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 96 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813587:0:10850:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 96 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813596:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 96 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.813623:0:10850:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 96 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.813636:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668422656:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784827.813721:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784827.818739:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.818808:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658194560 00000100:00100000:2.0:1787784827.818817:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658194560 00000100:00100000:2.0:1787784827.818879:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 246 00000100:00100000:2.0:1787784827.818889:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30128:x1874627658194560:12345-192.168.203.43@tcp:49:cat.0 00000100:00100000:2.0:1787784827.819211:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30128:x1874627658194560:12345-192.168.203.43@tcp:49:cat.0 Request processed in 323us (479us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784827.819220:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 246 00000100:00100000:1.0:1787784827.823557:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668422656:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784827.823592:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823604:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989500031000/0xb0ce63137acc6a1f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823653:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823668:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823679:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823689:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823696:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823702:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823711:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.823715:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823721:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823733:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:1.0:1787784827.823736:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784827.823740:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823746:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6af8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823756:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823762:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823768:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823773:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784827.823776:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823782:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823793:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:1.0:1787784827.823795:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784827.823798:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 2/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.823803:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6aea lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.824218:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+52:30129:x1874627658194304:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 27503us (27694us total) trans 4294968112 rc 0/0 00000100:00100000:1.0:1787784827.824229:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1232 00000020:00080000:1.0:1787784827.824235:0:10850:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:3.0:1787784827.828537:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784827.828547:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784827.829478:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:3.0:1787784827.829489:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784827.829659:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.829826:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658194816 00000100:00100000:3.0:1787784827.829835:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658194816 00000100:00100000:3.0:1787784827.830086:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 247 00000100:00100000:3.0:1787784827.830097:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:30129:x1874627658194816:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1787784827.831219:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x24:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1787784827.831412:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968113 00010000:00010000:3.0:1787784827.831466:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.831482:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.831494:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.831503:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.831509:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.831515:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.831586:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784827.831741:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30129:x1874627658194816:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1645us (2095us total) trans 4294968114 rc 0/0 00000100:00100000:3.0:1787784827.831750:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 247 00000020:00080000:3.0:1787784827.831756:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1787784827.834044:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.834227:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658194944 00000100:00100000:1.0:1787784827.834236:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658194944 00010000:00010000:2.0:1787784827.834273:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784827.834284:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.834306:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784827.834311:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.834320:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.834335:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff9895198b5700) 00000100:00100000:1.0:1787784827.834368:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 339 00000100:00100000:1.0:1787784827.834378:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:30128:x1874627658194944:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784827.834393:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784827.834411:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1339 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784827.834436:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 158, mtime: 0x6a8f6e78, atime: 0x6a8f6e79, ctime: 0x6a8f6e78, blocks: 0xb 00010000:00010000:1.0:1787784827.834455:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=618s v=(0 0) 00010000:00010000:1.0:1787784827.834459:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784827.834468:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 17 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.834483:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 17 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.834497:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c600, rc 0) 00000100:00100000:1.0:1787784827.834539:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:30128:x1874627658194944:12345-192.168.203.43@tcp:101:cat.0 Request processed in 163us (505us total) trans 0 rc 0/0 00010000:00010000:2.0:1787784827.834547:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00000100:00100000:1.0:1787784827.834548:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 339 00010000:00010000:2.0:1787784827.834555:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 1/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.834563:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6aff lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.834832:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787784827.837183:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787784827.837195:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784827.837198:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784827.837200:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784827.837568:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.837872:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658195072 00000100:00100000:2.0:1787784827.837887:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658195072 00002000:00100000:2.0:1787784827.837934:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x18:0x0] (0->4095) req@ffff9894035ac3c0 x1874627658195072/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:708/0 lens 488/0 e 0 to 0 dl 1787784838 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784827.837945:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894035ac3c0 with x1874627658195072 ext(0->4095) 00002000:00010000:2.0:1787784827.837950:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894035ac3c0 00000100:00100000:2.0:1787784827.837981:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 108 00000100:00100000:2.0:1787784827.837991:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658195072:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:0.0:1787784827.838865:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658195072:12345-192.168.203.43@tcp:3:cat.0 Request processed in 872us (1297us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784827.838883:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 108 00002000:00100000:0.0:1787784827.838892:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x18:0x0] (0->4095) req@ffff9894035ac3c0 x1874627658195072/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:708/0 lens 488/4536 e 0 to 0 dl 1787784838 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784827.838901:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894035ac3c0 with x1874627658195072 ext(0->4095) 00002000:00010000:0.0:1787784827.838906:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894035ac3c0 00000100:00100000:3.0:1787784827.844666:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.844732:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658195328 00000100:00100000:2.0:1787784827.844741:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658195328 00000100:00100000:2.0:1787784827.844803:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 248 00000100:00100000:2.0:1787784827.844813:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30128:x1874627658195328:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784827.845606:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30128:x1874627658195328:12345-192.168.203.43@tcp:35:cat.0 Request processed in 791us (946us total) trans 4294968115 rc 0/0 00000100:00100000:2.0:1787784827.845616:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 248 00000100:00100000:3.0:1787784827.908389:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784827.908602:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658195456 00000100:00100000:0.0:1787784827.908610:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658195456 00000100:00100000:0.0:1787784827.908784:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1233 00000100:00100000:0.0:1787784827.908796:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30130:x1874627658195456:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784827.908821:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784827.908841:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6b0d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1340 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.908907:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.908922:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.908932:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.908959:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.908967:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.908974:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909309:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909320:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909328:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784827.909358:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:0.0:1787784827.909362:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784827.909380:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909391:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909399:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909404:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909410:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909417:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784827.909421:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909429:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909440:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784827.909443:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784827.909447:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909458:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6b14 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909469:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909474:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909479:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909483:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784827.909486:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909490:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909496:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff989520108400) 00010000:00010000:0.0:1787784827.909498:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784827.909501:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909505:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6b1b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909535:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6b0d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909565:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784827.909572:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:0.0:1787784827.909575:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784827.909577:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:0.0:1787784827.909636:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30130:x1874627658195456:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 843us (1257us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784827.914218:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1233 00000100:00100000:1.0:1787784827.943131:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.944415:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658195584 00000100:00100000:3.0:1787784827.944426:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658195584 00000100:00100000:3.0:1787784827.944492:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1234 00000100:00100000:3.0:1787784827.944502:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30130:x1874627658195584:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784827.944521:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784827.944542:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b29 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1347 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.944620:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.944635:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.944648:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784827.944846:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 7680 00000004:00200000:3.0:1787784827.944850:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784827.944940:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.944969:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b29 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.944986:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 100 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.944995:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:3.0:1787784827.944998:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784827.945002:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784827.945066:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30130:x1874627658195584:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 566us (1944us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784827.945073:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1234 00000100:00100000:1.0:1787784827.962514:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.962618:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658195840 00000100:00100000:2.0:1787784827.962628:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658195840 00000100:00100000:2.0:1787784827.962687:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1235 00000100:00100000:2.0:1787784827.962698:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30130:x1874627658195840:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784827.962728:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784827.962752:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6b37 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b134e expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.962849:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b3e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.962863:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b3e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.962874:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b3e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784827.962885:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:2.0:1787784827.963026:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6b37 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b134e expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.963045:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b3e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b134e expref: 102 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784827.963053:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:2.0:1787784827.963056:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1787784827.963059:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784827.963123:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30130:x1874627658195840:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 426us (618us total) trans 4294968116 rc 0/0 00000100:00100000:2.0:1787784827.963130:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1235 00000100:00100000:0.0:1787784827.968923:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784827.969046:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658195968 00000100:00100000:1.0:1787784827.969055:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658195968 00000100:00100000:1.0:1787784827.969115:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:1.0:1787784827.969125:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658195968:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:1.0:1787784827.969406:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.969427:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 17 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.969436:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1787784827.969441:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784827.969445:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 4/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.969480:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 17 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.969494:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 17 pid: 6669 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.969508:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbfc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668422784:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784827.969834:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784827.973227:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.973366:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658196096 00000100:00100000:3.0:1787784827.973376:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658196096 00000100:00100000:1.0:1787784827.973402:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbfc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668422784:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784827.973417:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 18 pid: 6669 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.973428:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.973501:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 370 00000100:00100000:3.0:1787784827.973512:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2356:x1874627658196096:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784827.973528:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784827.973551:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 18 pid: 6669 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.973574:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 18 pid: 6669 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.973601:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d75fd59/0xb0ce63137acc6b06 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1339 expref: 18 pid: 6669 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.973616:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784827.973621:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784827.973626:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.973637:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.973666:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784827.973728:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658196096:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 217us (507us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784827.973736:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 370 00010000:00010000:1.0:1787784827.973932:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.973952:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.973976:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.973984:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.973992:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.973999:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784827.974004:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.974129:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.974142:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784827.974236:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090185ac7/0xb0ce63137acc6b4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784827.974352:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658195968:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 5227us (5437us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784827.974361:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:0.0:1787784827.984625:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784827.984797:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658196352 00000100:00100000:3.0:1787784827.984807:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658196352 00000100:00100000:3.0:1787784827.984861:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 249 00000100:00100000:3.0:1787784827.984879:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d5c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30130:x1874627658196352:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:3.0:1787784827.984939:0:5815:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x23:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b3e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b134e expref: 102 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784827.985349:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968117 00000001:00080000:3.0:1787784827.988129:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968118 00010000:00010000:3.0:1787784827.988245:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.988268:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.988279:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 105 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.988309:0:5815:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.988348:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 105 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.988364:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 105 pid: 5811 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.988378:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be580 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668422912:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784827.988480:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784827.988489:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 105 pid: 10850 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784827.988498:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668423040:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784827.988521:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784827.999852:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784827.999892:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784827.999984:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658196480 00000100:00100000:2.0:1787784827.999994:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658196480 00010000:00010000:2.0:1787784828.000033:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 106 pid: 5811 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784828.000072:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658196608 00000100:00100000:2.0:1787784828.000076:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658196608 00010000:00010000:2.0:1787784828.000089:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 107 pid: 10850 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784828.000112:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 371 00000100:00100000:2.0:1787784828.000122:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2357:x1874627658196480:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784828.000137:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784828.000144:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 107 pid: 5811 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000196:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 107 pid: 5811 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000218:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:2.0:1787784828.000224:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000233:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000238:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784828.000284:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2357:x1874627658196480:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 163us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.000305:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 371 00000100:00100000:2.0:1787784828.000349:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 372 00000100:00100000:2.0:1787784828.000369:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658196608:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784828.000377:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784828.000382:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 106 pid: 10850 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000393:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 106 pid: 10850 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000404:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:2.0:1787784828.000408:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000420:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000427:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.000445:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784828.000464:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658196608:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 108us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.000471:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 372 00000100:00080000:2.0:1787784828.000488:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784828.001011:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6b22 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1340 expref: 105 pid: 5811 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784828.001028:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be580 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668422912:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784828.001040:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fce00/0xb0ce63137acc6a26 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b12f3 expref: 104 pid: 10850 timeout: 719 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784828.001055:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668423040:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784828.001076:0:5815:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.001085:0:5815:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.001090:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784828.002569:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968119 00000001:00080000:2.0:1787784828.002806:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968120 00000001:00080000:2.0:1787784828.002867:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968121 00000100:00100000:2.0:1787784828.002977:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668423168:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1787784828.003041:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784828.003077:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9c180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784828.003273:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668423168 00000100:00100000:1.0:1787784828.003294:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668423168 00000100:00100000:1.0:1787784828.003350:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57 00000100:00100000:1.0:1787784828.003358:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668423168:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1787784828.003437:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668423168:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 79us (395us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784828.003444:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57 00000100:00100000:2.0:1787784828.003781:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668423168:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:2.0:1787784828.004117:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968122 00000100:00100000:2.0:1787784828.004461:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668423296:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1787784828.004499:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784828.004563:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1480 going to sleep for 11 seconds 00000100:00100000:3.0:1787784828.008176:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668423296 00000100:00100000:3.0:1787784828.008703:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668423296 00000100:00100000:1.0:1787784828.011300:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 58 00000100:00100000:1.0:1787784828.011315:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668423296:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1787784828.011575:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:5815:x1874627668423296:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 262us (7074us total) trans 4294967363 rc 0/0 00000100:00100000:1.0:1787784828.011583:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 58 00000100:00100000:2.0:1787784828.011665:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668423296:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:2.0:1787784828.011696:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.011705:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.011713:0:5815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.011719:0:5815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784828.011724:0:5815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.011733:0:5815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.011748:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:2.0:1787784828.011751:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1787784828.011756:0:5815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.011762:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6b53 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784828.011985:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x4a:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1787784828.012085:0:5815:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].10, 64 off8856 00000004:00080000:2.0:1787784828.012097:0:5815:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 619098948745ns, next at 620098946217ns, rc = 0 00000001:00080000:2.0:1787784828.012103:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968123 00010000:00010000:2.0:1787784828.012175:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff989520108000) 00010000:00010000:2.0:1787784828.012179:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1787784828.012185:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b3e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b134e expref: 108 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784828.012280:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d5c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:30130:x1874627658196352:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 27410us (27662us total) trans 4294968124 rc 0/0 00000100:00100000:2.0:1787784828.012300:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 249 00000020:00080000:2.0:1787784828.012304:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:0.0:1787784828.120964:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784828.121044:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658196864 00000100:00100000:3.0:1787784828.121054:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658196864 00000100:00100000:3.0:1787784828.121111:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1236 00000100:00100000:3.0:1787784828.121122:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:30136:x1874627658196864:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784828.121188:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784828.121210:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b5a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1355 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121306:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121321:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121332:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121351:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121359:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121365:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121697:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6b6f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121707:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6b6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121714:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6b6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784828.121724:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:3.0:1787784828.121810:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121818:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121835:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121841:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.121846:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121857:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121870:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:3.0:1787784828.121873:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784828.121878:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121884:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b61 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121896:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121902:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121908:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121913:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.121916:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121922:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121929:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff9895198b5b00) 00010000:00010000:3.0:1787784828.121931:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784828.121934:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121945:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.121989:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b5a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1355 expref: 110 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.122003:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6b6f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1355 expref: 109 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.122010:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5400) 00010000:00010000:3.0:1787784828.122013:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784828.122015:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784828.122078:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:30136:x1874627658196864:12345-192.168.203.43@tcp:101:cat.0 Request processed in 958us (1124us total) trans 4294968125 rc 0/0 00000100:00100000:3.0:1787784828.122086:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1236 00000100:00100000:3.0:1787784828.131230:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784828.131754:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658197120 00000100:00100000:2.0:1787784828.131764:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658197120 00000100:00100000:2.0:1787784828.131837:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 55 00000100:00100000:2.0:1787784828.131847:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:30136:x1874627658197120:12345-192.168.203.43@tcp:103:cat.0 00010000:00010000:2.0:1787784828.131863:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784828.131887:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6b6f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1355 expref: 109 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.131907:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5400) 00010000:00010000:2.0:1787784828.131917:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784828.132038:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30136:x1874627658197120:12345-192.168.203.43@tcp:103:cat.0 Request processed in 192us (816us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.132046:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 55 00000100:00100000:0.0:1787784828.138724:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784828.138908:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658197248 00000100:00100000:3.0:1787784828.138918:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658197248 00000100:00100000:3.0:1787784828.138977:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1237 00000100:00100000:3.0:1787784828.138987:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30136:x1874627658197248:12345-192.168.203.43@tcp:58:cat.0 00010000:00010000:3.0:1787784828.139097:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.139112:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.139124:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784828.139579:0:20438:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].36, 136 off13088 00000100:00100000:3.0:1787784828.139668:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30136:x1874627658197248:12345-192.168.203.43@tcp:58:cat.0 Request processed in 681us (962us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784828.139677:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1237 00000100:00100000:1.0:1787784828.151648:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784828.154045:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658197376 00000100:00100000:2.0:1787784828.154058:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658197376 00000100:00100000:2.0:1787784828.154127:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1238 00000100:00100000:2.0:1787784828.154137:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30136:x1874627658197376:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784828.154180:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784828.154203:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b135c expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.154267:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.154279:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b135c expref: 109 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.154528:0:5813:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.154542:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b135c expref: 109 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.154554:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784828.154557:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784828.154561:0:5813:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b135c expref: 109 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.154568:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fc600, rc 0) 00000100:00100000:2.0:1787784828.154632:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:30136:x1874627658197376:12345-192.168.203.43@tcp:101:cat.0 Request processed in 497us (2993us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.154639:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1238 00000100:00100000:3.0:1787784828.401483:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784828.401514:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668423424:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784828.401583:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784828.401904:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668423424 00000100:00100000:0.0:1787784828.401912:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668423424 00000100:00100000:0.0:1787784828.401964:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1787784828.401974:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3279:x1874627668423424:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784828.401995:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3857711104 avail 3854565376 00000100:00100000:0.0:1787784828.402048:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3279:x1874627668423424:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (464us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784828.402055:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:3.0:1787784828.402627:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668423424:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784828.682227:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.682269:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.682310:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.682357:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.682376:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.682401:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.682426:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.682451:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658197888 00000100:00100000:0.0:1787784828.682459:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658197888 00000100:00100000:0.0:1787784828.682510:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658198144 00000100:00100000:0.0:1787784828.682514:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658198144 00000100:00100000:0.0:1787784828.682527:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658197760 00000100:00100000:0.0:1787784828.682530:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658197760 00000100:00100000:0.0:1787784828.682540:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658198272 00000100:00100000:0.0:1787784828.682542:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658198272 00000100:00100000:0.0:1787784828.682556:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 388 00000100:00100000:0.0:1787784828.682565:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658197888:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784828.682758:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 389 00000100:00100000:2.0:1787784828.682772:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658198144:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784828.682832:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658197504 00000100:00100000:3.0:1787784828.682837:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658197504 00000100:00100000:3.0:1787784828.682869:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 577 00000100:00100000:3.0:1787784828.682877:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658197504:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784828.682957:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 390 00000100:00100000:1.0:1787784828.682968:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658197760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784828.694209:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658198144:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 11432us (11887us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.694223:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 389 00000100:00100000:2.0:1787784828.694266:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:2.0:1787784828.694292:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658198272:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784828.694338:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658198272:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 64us (11910us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.694348:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:2.0:1787784828.694470:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658197632 00000100:00100000:2.0:1787784828.694478:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658197632 00000100:00100000:2.0:1787784828.694521:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658198016 00000100:00100000:2.0:1787784828.694525:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658198016 00000100:00100000:2.0:1787784828.694554:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:2.0:1787784828.694561:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:2354:x1874627658197632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784828.694593:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:2354:x1874627658197632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (12376us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.694599:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:2.0:1787784828.694613:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 431 00000100:00100000:2.0:1787784828.694620:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658198016:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784828.694647:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658198016:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (12246us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.694653:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 431 00000100:00080000:2.0:1787784828.694710:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784828.695167:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658197504:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 12289us (12808us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784828.695176:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 577 00000100:00100000:0.0:1787784828.696237:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:2354:x1874627658197888:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 13666us (13961us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784828.696249:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 388 00000100:00100000:1.0:1787784828.699767:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658197760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 16791us (17378us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784828.699782:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 390 00000040:00080000:3.0:1787784828.849678:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784828.849693:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784828.849737:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 37 count 4 00000040:00080000:3.0:1787784828.849745:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784828.849945:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668423552:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784828.850071:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4b40 going to sleep for 11 seconds 00010000:00010000:2.0:1787784828.851113:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784828.854368:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668423552:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784828.859504:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784828.859675:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658198400 00000100:00100000:2.0:1787784828.859687:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658198400 00000100:00100000:2.0:1787784828.859771:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1239 00000100:00100000:2.0:1787784828.859783:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30137:x1874627658198400:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784828.859847:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784828.860015:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30137:x1874627658198400:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 234us (524us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.860024:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1239 00000100:00100000:2.0:1787784828.866629:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784828.866990:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658198528 00000100:00100000:3.0:1787784828.867004:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658198528 00000100:00100000:3.0:1787784828.868490:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1240 00000100:00100000:3.0:1787784828.870382:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30137:x1874627658198528:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1787784828.870499:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870521:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870538:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870559:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870567:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870574:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870601:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870610:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870615:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784828.870662:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:3.0:1787784828.870667:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784828.870683:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870696:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870702:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870708:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870715:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.870719:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870727:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870741:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784828.870744:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784828.870750:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870756:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6b8b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870776:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870782:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870788:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870795:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.870798:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870804:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870812:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x92f5931a (ffff9895198b5c00) 00010000:00010000:3.0:1787784828.870815:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784828.870818:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870823:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6b92 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870838:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870845:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870851:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870856:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.870858:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870864:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870876:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:3.0:1787784828.870878:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784828.870881:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.870887:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6b99 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784828.870997:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30137:x1874627658198528:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 2490us (4716us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784828.871007:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1240 00000100:00100000:1.0:1787784828.891231:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784828.891714:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658198784 00000100:00100000:2.0:1787784828.891738:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658198784 00000100:00100000:2.0:1787784828.891931:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1241 00000100:00100000:2.0:1787784828.891947:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30137:x1874627658198784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784828.891979:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784828.892006:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6ba0 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1363 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893389:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893418:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893432:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784828.893446:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:2.0:1787784828.893580:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893588:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893595:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893603:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784828.893607:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893621:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893635:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784828.893639:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784828.893644:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893651:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6ba7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893708:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6ba0 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1363 expref: 56 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893720:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784828.893722:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784828.893726:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6ba0 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1363 expref: 56 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.893737:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784828.893825:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+55:30137:x1874627658198784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1882us (2603us total) trans 4294968126 rc 301/301 00000100:00100000:2.0:1787784828.893836:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1241 00000100:00100000:0.0:1787784828.901515:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784828.908954:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658198912 00000100:00100000:1.0:1787784828.908973:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658198912 00000100:00100000:1.0:1787784828.909186:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1242 00000100:00100000:1.0:1787784828.909199:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:30137:x1874627658198912:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784828.909244:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784828.909389:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:30137:x1874627658198912:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 192us (7886us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784828.909397:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1242 00000100:00100000:2.0:1787784828.939560:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784828.939738:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658199168 00000100:00100000:0.0:1787784828.939751:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658199168 00000100:00100000:0.0:1787784828.939820:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1243 00000100:00100000:0.0:1787784828.939831:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+55:30137:x1874627658199168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784828.939858:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784828.939893:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6bb5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b136a expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784828.940861:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6bbc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784828.940894:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6bbc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784828.940909:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6bbc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784828.940922:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x25:0x0]: returning size 0 00010000:00010000:0.0:1787784828.940972:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6bb5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b136a expref: 58 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784828.940993:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6bbc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b136a expref: 57 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784828.941002:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff989520108400) 00010000:00010000:0.0:1787784828.941005:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784828.941009:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2600, rc 0) 00000100:00100000:0.0:1787784828.941094:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+57:30137:x1874627658199168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1264us (1558us total) trans 4294968127 rc 0/0 00000100:00100000:0.0:1787784828.941103:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1243 00000100:00100000:0.0:1787784828.964910:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784828.965091:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658199424 00000100:00100000:1.0:1787784828.965105:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658199424 00000100:00100000:1.0:1787784828.965296:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1244 00000100:00100000:1.0:1787784828.965309:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:30137:x1874627658199424:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784828.965382:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784828.965414:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6bbc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b136a expref: 57 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784828.965439:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff989520108400) 00010000:00010000:1.0:1787784828.965456:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784828.965485:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784828.965501:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 108 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784828.965510:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 4/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784828.965562:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 108 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784828.965579:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 108 pid: 20438 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784828.965595:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668423680:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784828.965705:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784828.972253:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784828.972299:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658199552 00000100:00100000:3.0:1787784828.972309:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658199552 00010000:00010000:3.0:1787784828.972355:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 109 pid: 20438 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784828.972395:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 373 00000100:00100000:3.0:1787784828.972405:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:2356:x1874627658199552:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784828.972422:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784828.972430:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 109 pid: 20438 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972446:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 109 pid: 20438 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972474:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:3.0:1787784828.972481:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972493:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972502:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972524:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784828.972591:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:2356:x1874627658199552:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 187us (350us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784828.972599:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 373 00010000:00010000:3.0:1787784828.972653:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6b30 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1347 expref: 108 pid: 20438 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784828.972861:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668423680:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784828.972888:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972898:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972906:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972930:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972940:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.972953:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973719:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973732:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973740:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973747:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.973752:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973761:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973773:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:3.0:1787784828.973776:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784828.973780:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973787:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6bca lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973799:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973805:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973812:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973818:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.973821:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973826:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973832:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x5e02fa06 (ffff9895198b5c00) 00010000:00010000:3.0:1787784828.973835:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784828.973837:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.973843:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bd1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x5e02fa06 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784828.973929:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+56:30137:x1874627658199424:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 8622us (9031us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784828.973938:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1244 00000100:00100000:3.0:1787784828.977856:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784828.977896:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658199680 00000100:00100000:3.0:1787784828.977907:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658199680 00000100:00100000:3.0:1787784828.977969:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1245 00000100:00100000:3.0:1787784828.977980:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:30137:x1874627658199680:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784828.978004:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784828.978030:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6bd8 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1371 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.978071:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 3/1,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.978087:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.978100:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.978250:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6bd8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 58 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.978271:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 57 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.978841:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784828.978845:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784828.978849:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784828.978918:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+57:30137:x1874627658199680:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 939us (1072us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784828.978926:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1245 00000100:00100000:3.0:1787784828.982635:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784828.982731:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658199936 00000100:00100000:2.0:1787784828.982743:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658199936 00000100:00100000:2.0:1787784828.982808:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1246 00000100:00100000:2.0:1787784828.982819:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+57:30137:x1874627658199936:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784828.982934:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.982955:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.982968:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784828.983247:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c4ff1d83 reply state 000000007d723d6d transno 4294968128 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784828.983258:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784828.983379:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+59:30137:x1874627658199936:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 561us (755us total) trans 4294968128 rc 0/0 00000100:00100000:2.0:1787784828.983387:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1246 00010000:00010000:3.0:1787784828.985310:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.985327:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.985341:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.985348:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784828.985353:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.985362:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.985382:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784828.985385:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787784828.985391:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784828.985398:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6be6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784828.986897:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784828.989443:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658200064 00000100:00100000:2.0:1787784828.989462:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658200064 00000100:00100000:2.0:1787784828.989544:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1247 00000100:00100000:2.0:1787784828.989555:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+58:30137:x1874627658200064:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784828.989586:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784828.989611:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 2/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1378 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.989689:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.989705:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784828.989721:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784828.989724:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784828.989729:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784828.989802:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+59:30137:x1874627658200064:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 249us (2916us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784828.989810:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1247 00000100:00100000:2.0:1787784828.995504:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784828.995650:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658200192 00000100:00100000:3.0:1787784828.995663:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658200192 00000100:00100000:3.0:1787784828.995728:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 250 00000100:00100000:3.0:1787784828.995739:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+59:30137:x1874627658200192:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1787784828.995908:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+59:30137:x1874627658200192:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 170us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784828.995915:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 250 00000001:00080000:3.0:1787784829.001858:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967371 is committed 00000001:00080000:3.0:1787784829.001887:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967370 is committed 00000001:00080000:3.0:1787784829.001898:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967372 is committed 00000100:00100000:3.0:1787784829.004400:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784829.004521:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658200448 00000100:00100000:2.0:1787784829.004533:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658200448 00000100:00100000:2.0:1787784829.004635:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 251 00000100:00100000:2.0:1787784829.004647:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+59:30137:x1874627658200448:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784829.005016:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:30137:x1874627658200448:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 370us (628us total) trans 4294968129 rc 0/0 00000100:00100000:2.0:1787784829.005024:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 251 00000100:00080000:0.0:1787784829.005557:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784829.010460:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784829.010967:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658200704 00000100:00100000:1.0:1787784829.010980:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658200704 00000100:00100000:1.0:1787784829.011366:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1248 00000100:00100000:0.0:1787784829.011474:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:30137:x1874627658200704:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784829.011653:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:30137:x1874627658200704:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 277us (1206us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784829.011663:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1248 00000100:00100000:2.0:1787784829.015484:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.015561:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658200960 00000100:00100000:3.0:1787784829.015573:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658200960 00000100:00100000:3.0:1787784829.015635:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1249 00000100:00100000:3.0:1787784829.015646:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+60:30137:x1874627658200960:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784829.015670:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784829.015695:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bf4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b137f expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.015820:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.015838:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 61 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.015848:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 4/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.015901:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 61 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.015918:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 61 pid: 5813 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.015934:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530505a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668423808:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784829.016518:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895067c3780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784829.020308:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784829.020357:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658201088 00000100:00100000:2.0:1787784829.020367:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658201088 00010000:00010000:2.0:1787784829.020414:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 62 pid: 5813 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784829.020454:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:2.0:1787784829.020466:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+62:2357:x1874627658201088:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784829.020484:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784829.020492:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 62 pid: 5813 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.020507:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 62 pid: 5813 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.020534:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784829.020540:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.021022:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc6bed lrc: 0/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1378 expref: 62 pid: 5813 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784829.021053:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530505a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668423808:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784829.021099:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.021109:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.021704:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.021850:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784829.021931:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.021939:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784829.021962:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x25:0x0]: returning size 0 00000004:00080000:0.0:1787784829.022511:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1b:0x0], index 1: 0 00010000:00010000:0.0:1787784829.022776:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022786:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022794:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022800:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784829.022804:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022812:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022827:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:0.0:1787784829.022830:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784829.022835:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022842:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc6bfb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022964:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bf4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b137f expref: 62 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022977:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff98950a38c400) 00010000:00010000:0.0:1787784829.022979:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784829.022983:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc6bf4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b137f expref: 62 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.022994:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7400, rc 0) 00000100:00100000:0.0:1787784829.023264:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+61:30137:x1874627658200960:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 7618us (7790us total) trans 4294968130 rc 301/301 00000100:00100000:0.0:1787784829.023289:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1249 00000100:00100000:2.0:1787784829.023486:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+60:2357:x1874627658201088:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 3020us (3189us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784829.023496:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:2.0:1787784829.026355:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784829.027095:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658201344 00000100:00100000:0.0:1787784829.027107:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658201344 00000100:00100000:0.0:1787784829.027191:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 252 00000100:00100000:0.0:1787784829.027201:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:30137:x1874627658201344:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784829.028043:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+61:30137:x1874627658201344:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 831us (1688us total) trans 4294968131 rc 0/0 00000100:00100000:0.0:1787784829.028053:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 252 00000100:00100000:2.0:1787784829.038505:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.039994:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658201472 00000100:00100000:3.0:1787784829.040011:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658201472 00000100:00100000:3.0:1787784829.040259:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1250 00000100:00100000:3.0:1787784829.040272:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+61:30137:x1874627658201472:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784829.040323:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784829.040349:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c09 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1386 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.040439:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c10 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.040457:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c10 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.040471:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c10 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784829.040514:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x25:0x0]: returning size 0 00000004:00200000:3.0:1787784829.040519:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784829.040541:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c10 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.040569:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c09 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1386 expref: 63 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.040592:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c10 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1386 expref: 62 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.040602:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784829.040605:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1787784829.040609:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784829.040683:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+62:30137:x1874627658201472:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 414us (2192us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784829.040691:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1250 00000100:00100000:0.0:1787784829.054431:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784829.054592:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658201600 00000100:00100000:1.0:1787784829.054604:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658201600 00000100:00100000:1.0:1787784829.054682:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:1.0:1787784829.054695:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658201600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784829.054718:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784829.054746:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b138d expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784829.054780:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784829.054805:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=620s v=(0 0) 00010000:00010000:1.0:1787784829.054810:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784829.054822:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.054845:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.054861:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784829.055220:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658201600:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 526us (803us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784829.055228:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:2.0:1787784829.059261:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784829.059386:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658201856 00000100:00100000:0.0:1787784829.059397:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658201856 00000100:00100000:0.0:1787784829.059462:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1251 00000100:00100000:0.0:1787784829.059474:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+62:30137:x1874627658201856:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784829.059964:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+62:30137:x1874627658201856:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 490us (714us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784829.059973:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1251 00000100:00100000:0.0:1787784829.078064:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784829.078185:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658201984 00000100:00100000:2.0:1787784829.078197:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658201984 00000100:00100000:2.0:1787784829.078264:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 342 00000100:00100000:2.0:1787784829.078275:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:30137:x1874627658201984:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784829.078308:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784829.078333:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1394 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.078374:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.078386:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784829.078391:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784829.078397:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.078441:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.078457:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 12 pid: 6669 timeout: 720 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784829.078473:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668423936:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784829.078590:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954051a240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784829.081871:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.081947:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658202112 00000100:00100000:3.0:1787784829.081958:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658202112 00010000:00010000:3.0:1787784829.082009:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 13 pid: 6669 timeout: 720 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.082068:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 375 00000100:00100000:3.0:1787784829.082078:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658202112:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784829.082096:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784829.082123:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 13 pid: 6669 timeout: 720 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.082183:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 13 pid: 6669 timeout: 720 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.082216:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784829.082220:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784829.082229:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.082241:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.082296:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.082314:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668424064:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784829.082405:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895067c3780 going to sleep for 11 seconds 00010000:00010000:2.0:1787784829.083752:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4594048/0xb0ce63137acc6c17 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b138d expref: 13 pid: 6669 timeout: 720 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784829.083780:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668423936:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784829.083810:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.083830:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784829.083899:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:30137:x1874627658201984:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5625us (5848us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784829.083910:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 342 00000100:00100000:0.0:1787784829.092908:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.092927:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668424064:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784829.092961:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784829.092999:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658202240 00000100:00100000:2.0:1787784829.093008:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658202240 00002000:00100000:2.0:1787784829.093065:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1b:0x0] (0->157) req@ffff989539101e00 x1874627658202240/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:710/0 lens 648/0 e 0 to 0 dl 1787784840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784829.093077:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539101e00 with x1874627658202240 ext(0->157) 00002000:00010000:2.0:1787784829.093083:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539101e00 00000100:00100000:2.0:1787784829.093118:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 109 00000100:00100000:2.0:1787784829.093127:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658202240:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1787784829.093521:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658202112:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 11441us (11659us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784829.093534:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 375 00000100:00100000:2.0:1787784829.093772:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658202240:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 644us (876us total) trans 4294967373 rc 0/0 00000100:00100000:2.0:1787784829.093780:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 109 00002000:00100000:2.0:1787784829.093787:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1b:0x0] (0->157) req@ffff989539101e00 x1874627658202240/t4294967373(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:710/0 lens 648/448 e 0 to 0 dl 1787784840 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784829.093796:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989539101e00 with x1874627658202240 ext(0->157) 00002000:00010000:2.0:1787784829.093800:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989539101e00 00000100:00100000:0.0:1787784829.099605:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.099697:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658202368 00000100:00100000:3.0:1787784829.099710:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658202368 00000100:00100000:3.0:1787784829.099788:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1252 00000100:00100000:3.0:1787784829.099799:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+62:30137:x1874627658202368:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:1.0:1787784829.162811:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968131 is committed 00000020:00080000:1.0:1787784829.162853:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989503421a80 refcount 2 committed 0 result 0 batchid 26 00000020:00080000:1.0:1787784829.162859:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989402003600 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9895034216c0 00000020:00080000:1.0:1787784829.162917:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989402003b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff98953f95e800 00000001:00080000:1.0:1787784829.162943:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968124 is committed 00000004:00080000:1.0:1787784829.162948:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 12 committed 00000020:00080000:1.0:1787784829.162963:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989504d9ccc0 refcount 2 committed 0 result 0 batchid 27 00000020:00080000:1.0:1787784829.162968:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989518f69200 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989510bc06c0 00000020:00080000:1.0:1787784829.162971:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989518f69600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff98953f088800 00000020:00080000:1.0:1787784829.163187:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 25 batchid updated 00000001:00080000:1.0:1787784829.163199:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968125 is committed 00000020:00080000:1.0:1787784829.163221:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 27 committed batchid 25 00000020:00080000:1.0:1787784829.163456:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 25 00000100:00100000:1.0:1787784829.163594:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668424192:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787784829.163688:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784829.163939:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30137:x1874627658202368:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 64137us (64345us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784829.163952:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1252 00000100:00100000:1.0:1787784829.164668:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668424192 00000100:00100000:1.0:1787784829.164679:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668424192 00000100:00100000:1.0:1787784829.164744:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:1.0:1787784829.164756:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3279:x1874627668424192:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1787784829.164791:0:6671:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:24 count 1 00002000:00010000:1.0:1787784829.164831:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 158, mtime: 0x6a8f6e78, atime: 0x6a8f6e79, ctime: 0x6a8f6e78, blocks: 0xb 00010000:00010000:1.0:1787784829.164841:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.164873:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=620s v=(0 0) 00010000:00010000:1.0:1787784829.164878:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784829.164886:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.164905:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.164917:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.164925:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.164945:0:6671:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787784829.165546:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3279:x1874627668424192:12345-0@lo:6:osp-syn-0-0.0 Request processed in 791us (1856us total) trans 4294967364 rc 0/0 00000100:00100000:1.0:1787784829.165555:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00010000:00010000:1.0:1787784829.165581:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784829.165589:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.165598:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784829.165603:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.165612:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.165629:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.165636:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d21a9863/0xb0ce63137acc6c25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.165658:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784829.165661:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784829.165663:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787784829.165702:0:3279:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401cc30c0/2, rc 0, transno 4294967364 00000100:00100000:1.0:1787784829.165710:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668424192:0@lo:6:osp-syn-0-0.0 00000100:00080000:2.0:1787784829.165855:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784829.180939:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784829.181048:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658202496 00000100:00100000:1.0:1787784829.181068:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658202496 00000100:00100000:1.0:1787784829.181137:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 344 00000100:00100000:1.0:1787784829.181176:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658202496:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1787784829.298380:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658202496:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 117218us (117439us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784829.298398:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 344 00000001:00080000:3.0:1787784829.301114:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967373 is committed 00000100:00100000:0.0:1787784829.305472:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.307207:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658202752 00000100:00100000:3.0:1787784829.307225:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658202752 00000100:00100000:3.0:1787784829.307420:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1253 00000100:00100000:3.0:1787784829.307433:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30137:x1874627658202752:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784829.307507:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784829.307541:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c10 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1386 expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.307567:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784829.307572:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784829.307608:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.307627:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.307638:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784829.307849:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7db10c2 reply state 0000000020b1bd74 transno 4294968132 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.307858:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784829.308251:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30137:x1874627658202752:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 717us (2691us total) trans 4294968132 rc 0/0 00000100:00100000:3.0:1787784829.308260:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1253 00000100:00100000:0.0:1787784829.311348:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:2.0:1787784829.311714:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.311730:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.311740:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.311747:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784829.311752:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.311762:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.311782:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784829.311786:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784829.311791:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.311799:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc6c2c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.311966:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658202880 00000100:00100000:3.0:1787784829.311978:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658202880 00000100:00100000:3.0:1787784829.312474:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1254 00000100:00100000:3.0:1787784829.312486:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30137:x1874627658202880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784829.312515:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784829.312549:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b139b expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.312621:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.312636:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.312658:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784829.312663:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784829.312666:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784829.312737:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30137:x1874627658202880:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 254us (1401us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784829.312746:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1254 00000100:00100000:0.0:1787784829.316618:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784829.316706:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658203008 00000100:00100000:2.0:1787784829.316720:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658203008 00000100:00100000:2.0:1787784829.316794:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 345 00000100:00100000:2.0:1787784829.316806:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658203008:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1787784829.317168:0:10688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:2.0:1787784829.317173:0:10688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb mtime from disk: 0 -> 1787784824 00002000:00010000:2.0:1787784829.317176:0:10688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb atime from disk: 0 -> 1787784827 00002000:00010000:2.0:1787784829.317178:0:10688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb ctime from disk: 0 -> 1787784824 00002000:00010000:2.0:1787784829.317180:0:10688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1787784829.317260:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658203008:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 455us (654us total) trans 4294967374 rc 0/0 00000100:00100000:2.0:1787784829.317270:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 345 00000100:00100000:0.0:1787784829.320557:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784829.320646:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658203264 00000100:00100000:1.0:1787784829.320659:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658203264 00000100:00100000:1.0:1787784829.320729:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1255 00000100:00100000:1.0:1787784829.320741:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30137:x1874627658203264:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1787784829.320823:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.320843:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.320853:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 4/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.320892:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.320908:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 48 pid: 5812 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784829.320924:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668424320:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784829.321648:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989505495d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784829.324303:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668424320:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784829.324340:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784829.325531:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.325756:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658203392 00000100:00100000:3.0:1787784829.325765:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658203392 00010000:00010000:3.0:1787784829.325817:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 49 pid: 5812 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.325870:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 376 00000100:00100000:3.0:1787784829.325882:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2356:x1874627658203392:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784829.325900:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784829.325908:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 49 pid: 5812 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.325923:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 49 pid: 5812 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.325949:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6bdf lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1371 expref: 49 pid: 5812 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.325970:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784829.325976:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.325986:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.326000:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.326034:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784829.326113:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2356:x1874627658203392:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 233us (589us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784829.326121:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 376 00010000:00010000:1.0:1787784829.326646:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326658:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326698:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326705:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326712:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326718:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784829.326723:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326731:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326742:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:1.0:1787784829.326746:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787784829.326750:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.326757:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6c3a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784829.326821:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30137:x1874627658203264:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 6081us (6277us total) trans 0 rc -22/-22 00000100:00100000:1.0:1787784829.326830:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1255 00000100:00100000:0.0:1787784829.329620:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784829.329706:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658203520 00000100:00100000:1.0:1787784829.329719:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658203520 00000100:00100000:1.0:1787784829.329785:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 346 00000100:00100000:1.0:1787784829.329797:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658203520:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784829.329866:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.329897:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.329907:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784829.329912:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784829.329917:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.329960:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.329982:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 12 pid: 10688 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784829.330000:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668424448:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784829.330103:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989505495d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784829.332682:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.332794:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658203648 00000100:00100000:3.0:1787784829.332805:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658203648 00010000:00010000:3.0:1787784829.332850:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 13 pid: 10688 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.332903:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 377 00000100:00100000:3.0:1787784829.332913:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953070f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658203648:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784829.332929:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784829.332954:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 13 pid: 10688 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.332969:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 13 pid: 10688 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.332998:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784829.333003:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784829.333011:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.333029:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.333053:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784829.333114:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953070f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658203648:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 202us (453us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784829.333122:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 377 00010000:00010000:1.0:1787784829.333491:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c1e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1394 expref: 12 pid: 10688 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784829.333509:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668424448:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784829.333531:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333542:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333550:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333577:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333586:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333600:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333608:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784829.333612:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333622:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333633:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784829.333641:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d75fd59/0xb0ce63137acc6c41 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784829.333713:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658203520:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 3918us (4103us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784829.333722:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 346 00000100:00100000:0.0:1787784829.336535:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.336620:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658203904 00000100:00100000:3.0:1787784829.336630:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658203904 00000100:00100000:3.0:1787784829.336760:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1256 00000100:00100000:3.0:1787784829.336772:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30137:x1874627658203904:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784829.336873:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.336893:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.336904:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.337454:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.337467:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.337475:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 4/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.337512:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.337527:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 48 pid: 5811 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784829.337542:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668424576:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784829.337668:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f7aee40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784829.340667:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784829.340759:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658204032 00000100:00100000:0.0:1787784829.340770:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658204032 00010000:00010000:0.0:1787784829.340819:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 49 pid: 5811 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784829.340875:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 378 00000100:00100000:0.0:1787784829.340885:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2356:x1874627658204032:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784829.340903:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784829.340911:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 49 pid: 5811 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.340927:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 49 pid: 5811 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.340954:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:0.0:1787784829.340960:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.340970:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.340978:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.341001:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784829.341062:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2356:x1874627658204032:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 178us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784829.341071:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 378 00010000:00010000:2.0:1787784829.342348:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc6c33 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b139b expref: 48 pid: 5811 timeout: 720 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784829.342375:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668424576:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784829.342399:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.342410:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.342416:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784829.342654:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968133 00000001:00080000:2.0:1787784829.343037:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968134 00000001:00080000:2.0:1787784829.343091:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968135 00000100:00100000:2.0:1787784829.343441:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668424704:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787784829.343507:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784829.343543:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405d99c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784829.343606:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668424704 00000100:00100000:0.0:1787784829.343614:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668424704 00000100:00100000:0.0:1787784829.343699:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 59 00000100:00100000:0.0:1787784829.343711:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668424704:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787784829.343799:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668424704:12345-0@lo:1000:mdt00_001.0 Request processed in 91us (291us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784829.343806:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 59 00000100:00100000:0.0:1787784829.343853:0:5812:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967372 gen 2 00000004:00080000:0.0:1787784829.343859:0:5812:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967371 committed transno 4294967372 00000004:00080000:0.0:1787784829.343888:0:5812:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff98953f7b2580, transno 4294967372 00000100:00100000:0.0:1787784829.343902:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668424704:0@lo:1000:mdt00_001.0 00000001:00080000:0.0:1787784829.344318:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968136 00000100:00100000:0.0:1787784829.344375:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668424832:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787784829.344407:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784829.344431:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169e40 going to sleep for 11 seconds 00000100:00080000:2.0:1787784829.344646:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784829.344754:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668424832 00000100:00100000:3.0:1787784829.344760:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668424832 00000100:00100000:3.0:1787784829.344787:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 60 00000100:00100000:3.0:1787784829.344795:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668424832:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787784829.344985:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5812:x1874627668424832:12345-0@lo:1000:mdt00_001.0 Request processed in 189us (576us total) trans 4294967375 rc 0/0 00000100:00100000:3.0:1787784829.344992:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 60 00000100:00100000:2.0:1787784829.345550:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668424832:0@lo:1000:mdt00_001.0 00000001:00080000:2.0:1787784829.345630:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968137 00010000:00010000:2.0:1787784829.345663:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345674:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345686:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345692:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784829.345696:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345705:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345723:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784829.345727:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784829.345732:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345739:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc6c4f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345772:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345778:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345784:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345789:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784829.345793:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345803:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345810:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784829.345812:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784829.345820:0:5812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b135c expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345832:0:5812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b135c expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345842:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b135c expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.345869:0:5812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fc600/0xb0ce63137acc6b84 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b135c expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784829.345880:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668424960:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784829.345950:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405d9e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784829.349290:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668424960:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784829.349330:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349342:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6b7d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349382:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349399:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349411:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349422:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349428:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349435:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349441:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784829.349445:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349452:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349465:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784829.349469:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784829.349472:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349478:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6c56 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349486:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349492:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349497:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349502:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784829.349505:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349511:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349517:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff9895198b5b00) 00010000:00010000:2.0:1787784829.349519:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784829.349522:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.349528:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6c48 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784829.349780:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:30137:x1874627658203904:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 13008us (13255us total) trans 4294968138 rc 0/0 00000100:00100000:2.0:1787784829.349791:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1256 00000020:00080000:2.0:1787784829.349798:0:5812:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:3.0:1787784829.349953:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000040:00080000:0.0:1787784829.351586:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784829.351596:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784829.352108:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1787784829.352117:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784829.352519:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784829.352558:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658204160 00000100:00100000:3.0:1787784829.352567:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658204160 00000100:00100000:3.0:1787784829.352604:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658204416 00000100:00100000:3.0:1787784829.352606:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658204416 00000100:00100000:3.0:1787784829.352879:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 253 00000100:00100000:3.0:1787784829.352891:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30136:x1874627658204160:12345-192.168.203.43@tcp:49:cat.0 00000100:00100000:3.0:1787784829.353032:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30136:x1874627658204160:12345-192.168.203.43@tcp:49:cat.0 Request processed in 144us (3079us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784829.355702:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 253 00000100:00100000:3.0:1787784829.355746:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 254 00000100:00100000:3.0:1787784829.355762:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d4c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:30137:x1874627658204416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1787784829.356646:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x25:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1787784829.356820:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968139 00010000:00010000:3.0:1787784829.356923:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.356942:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.356956:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.356966:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.356973:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.356980:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784829.357008:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787784829.357190:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784829.357199:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.357207:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784829.357211:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.357220:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.357236:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff9895198b5c00) 00010000:00010000:0.0:1787784829.357239:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787784829.357244:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 1/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.357251:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6c5d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784829.357268:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784829.357270:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784829.357448:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784829.357583:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d4c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30137:x1874627658204416:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1823us (5064us total) trans 4294968140 rc 0/0 00000100:00100000:3.0:1787784829.357593:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 254 00000020:00080000:3.0:1787784829.357598:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1787784829.359410:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784829.359492:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658204544 00000100:00100000:2.0:1787784829.359504:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658204544 00000100:00100000:2.0:1787784829.359564:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:2.0:1787784829.359576:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+12:30136:x1874627658204544:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784829.359600:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784829.359630:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c64 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b13a2 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784829.359663:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1b:0x0] initial LVB size: 158, mtime: 0x6a8f6e78, atime: 0x6a8f6e7b, ctime: 0x6a8f6e78, blocks: 0xb 00010000:00010000:2.0:1787784829.359698:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=620s v=(0 0) 00010000:00010000:2.0:1787784829.359704:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784829.359716:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c64 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13a2 expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.359733:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000057ff88f9/0xb0ce63137acc6c64 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13a2 expref: 13 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784829.359748:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fce00, rc 0) 00000100:00100000:2.0:1787784829.359810:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:30136:x1874627658204544:12345-192.168.203.43@tcp:101:cat.0 Request processed in 237us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784829.359818:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:0.0:1787784829.362376:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784829.362459:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658204672 00000100:00100000:2.0:1787784829.362470:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658204672 00002000:00100000:2.0:1787784829.362528:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1b:0x0] (0->4095) req@ffff989530516d00 x1874627658204672/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:710/0 lens 488/0 e 0 to 0 dl 1787784840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784829.362540:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530516d00 with x1874627658204672 ext(0->4095) 00002000:00010000:2.0:1787784829.362547:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530516d00 00000100:00100000:2.0:1787784829.362614:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 110 00000100:00100000:2.0:1787784829.362626:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658204672:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:2.0:1787784829.362900:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2357:x1874627658204672:12345-192.168.203.43@tcp:3:cat.0 Request processed in 276us (532us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784829.362909:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 110 00002000:00100000:2.0:1787784829.362917:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1b:0x0] (0->4095) req@ffff989530516d00 x1874627658204672/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:710/0 lens 488/4536 e 0 to 0 dl 1787784840 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784829.362925:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530516d00 with x1874627658204672 ext(0->4095) 00002000:00010000:2.0:1787784829.362929:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530516d00 00000100:00100000:0.0:1787784829.366820:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784829.366997:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658204928 00000100:00100000:2.0:1787784829.367010:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658204928 00000100:00100000:2.0:1787784829.367072:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 255 00000100:00100000:2.0:1787784829.367083:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30136:x1874627658204928:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784829.367640:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30136:x1874627658204928:12345-192.168.203.43@tcp:35:cat.0 Request processed in 557us (967us total) trans 4294968141 rc 0/0 00000100:00100000:2.0:1787784829.367649:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 255 00010000:00010000:2.0:1787784829.879640:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784830.449505:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668425088:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784830.449603:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784830.449692:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784830.449699:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668425216:0@lo:400:kworker.0 00000100:00100000:3.0:1787784830.449731:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784830.449770:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425216 00000100:00100000:3.0:1787784830.449778:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425216 00000100:00100000:3.0:1787784830.449825:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 578 00000100:00100000:3.0:1787784830.449834:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668425216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784830.450869:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668425216:12345-0@lo:400:kworker.0 Request processed in 1021us (1122us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784830.450881:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 578 00000100:00100000:3.0:1787784830.450951:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668425216:0@lo:400:kworker.0 00000100:00100000:0.0:1787784830.453498:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425088 00000100:00100000:0.0:1787784830.453516:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425088 00000100:00100000:0.0:1787784830.453580:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1787784830.453591:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668425088:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784830.453622:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3857711104 avail 3854565376 00000100:00100000:1.0:1787784830.453874:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668425344:0@lo:400:kworker.0 00000100:00100000:2.0:1787784830.454105:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668425600:0@lo:400:kworker.0 00000100:00100000:1.0:1787784830.457584:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784830.457656:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784830.457672:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668425472:0@lo:400:kworker.0 00000100:00100000:1.0:1787784830.457786:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784830.457858:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425344 00000100:00100000:1.0:1787784830.457868:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425344 00000100:00100000:1.0:1787784830.457923:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425472 00000100:00100000:1.0:1787784830.457926:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425472 00000100:00100000:1.0:1787784830.457963:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1787784830.457973:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668425344:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784830.458101:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668425344:12345-0@lo:400:kworker.0 Request processed in 129us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784830.458109:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1787784830.458125:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:1.0:1787784830.458132:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668425472:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784830.458181:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668425472:12345-0@lo:400:kworker.0 Request processed in 49us (395us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784830.458187:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00080000:1.0:1787784830.458208:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784830.458246:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668425472:0@lo:400:kworker.0 00000100:00100000:1.0:1787784830.458259:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668425344:0@lo:400:kworker.0 00000100:00100000:2.0:1787784830.458558:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784830.458601:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784830.458611:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668425728:0@lo:400:kworker.0 00000100:00100000:2.0:1787784830.458640:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784830.458671:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425728 00000100:00100000:2.0:1787784830.458675:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425728 00000100:00100000:2.0:1787784830.458775:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:2.0:1787784830.458782:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668425728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784830.458821:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668425728:12345-0@lo:400:kworker.0 Request processed in 40us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784830.458831:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1787784830.458850:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668425088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 5257us (9243us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784830.458873:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:2.0:1787784830.458876:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668425728:0@lo:400:kworker.0 00000100:00100000:2.0:1787784830.458904:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668425088:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784830.458980:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425600 00000100:00100000:1.0:1787784830.458984:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425600 00000100:00100000:1.0:1787784830.459090:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:1.0:1787784830.459098:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3276:x1874627668425600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784830.459168:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3276:x1874627668425600:12345-0@lo:400:kworker.0 Request processed in 49us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784830.459175:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00080000:1.0:1787784830.459190:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784830.459253:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668425600:0@lo:400:kworker.0 00010000:00010000:0.0:1787784830.899623:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784831.922388:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784832.442689:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784832.442697:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784832.442734:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 37 count 4 00000040:00080000:3.0:1787784832.442741:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784832.945744:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784833.095522:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967364 is committed 00000001:00080000:0.0:1787784833.095556:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967361 is committed 00000100:00100000:2.0:1787784833.521458:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784833.521491:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668425856:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784833.521588:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784833.523225:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425856 00000100:00100000:1.0:1787784833.523240:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425856 00000100:00100000:1.0:1787784833.524214:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1787784833.524227:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668425856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784833.524450:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3857711104 avail 3855613952 00000100:00080000:2.0:1787784833.524529:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784833.524611:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967364 gen 2 00000004:00080000:3.0:1787784833.524619:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967362 committed transno 4294967364 00000004:00080000:3.0:1787784833.524644:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967363 committed transno 4294967364 00000004:00080000:3.0:1787784833.524658:0:3279:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401cc30c0, transno 4294967364 00000100:00100000:3.0:1787784833.524688:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668425856:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784833.525421:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668425856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1193us (3830us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784833.525432:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1787784833.799689:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784833.799737:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784833.799769:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784833.799789:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784833.799813:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784833.799851:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784833.799888:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784833.799891:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205568 00000100:00100000:0.0:1787784833.799903:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205568 00000100:00100000:1.0:1787784833.799917:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205312 00000100:00100000:1.0:1787784833.799924:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205312 00000100:00100000:0.0:1787784833.799979:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205184 00000100:00100000:1.0:1787784833.799982:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205824 00000100:00100000:0.0:1787784833.799984:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205184 00000100:00100000:1.0:1787784833.799989:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205824 00000100:00100000:1.0:1787784833.800021:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205440 00000100:00100000:1.0:1787784833.800026:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205440 00000100:00100000:0.0:1787784833.800026:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 435 00000100:00100000:0.0:1787784833.800040:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658205568:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784833.800046:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205696 00000100:00100000:1.0:1787784833.800050:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205696 00000100:00100000:1.0:1787784833.800069:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 393 00000100:00100000:1.0:1787784833.800077:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658205312:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784833.800176:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658205312:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 99us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784833.800184:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 393 00000100:00100000:1.0:1787784833.800204:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 394 00000100:00100000:1.0:1787784833.800211:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658205824:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784833.800238:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658205824:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (448us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784833.800263:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 394 00000100:00100000:1.0:1787784833.800278:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:1.0:1787784833.800286:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658205440:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784833.800318:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658205568:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 280us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784833.800325:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658205440:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 36us (470us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784833.800331:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 435 00000100:00100000:1.0:1787784833.800339:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:1.0:1787784833.800361:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:0.0:1787784833.800365:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00100000:1.0:1787784833.800374:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658205696:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784833.800378:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:2356:x1874627658205184:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784833.800414:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658205696:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (523us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784833.800419:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:2356:x1874627658205184:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 43us (603us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784833.800429:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:0.0:1787784833.800437:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00080000:0.0:1787784833.800475:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784833.800480:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784833.800483:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784833.800922:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205056 00000100:00100000:3.0:1787784833.800928:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205056 00000100:00100000:3.0:1787784833.800972:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 579 00000100:00100000:3.0:1787784833.800983:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658205056:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784833.801027:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658205056:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 48us (1359us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784833.801035:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 579 00010000:00010000:0.0:1787784833.971025:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784834.571622:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784834.571674:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658205952 00000100:00100000:1.0:1787784834.571695:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658205952 00000100:00100000:1.0:1787784834.571775:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1257 00000100:00100000:1.0:1787784834.571786:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30097:x1874627658205952:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1787784834.571907:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:30097:x1874627658205952:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 122us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784834.571915:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1257 00000001:00080000:3.0:1787784834.660698:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967374 is committed 00000001:00080000:3.0:1787784834.660724:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967375 is committed 00000001:00080000:1.0:1787784834.674999:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968137 is committed 00000020:00080000:1.0:1787784834.675017:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98952a1696c0 refcount 2 committed 0 result 0 batchid 28 00000020:00080000:1.0:1787784834.675023:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989405138900 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98952a169b40 00000020:00080000:1.0:1787784834.675027:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989405138c00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff98953f050000 00000020:00080000:1.0:1787784834.675052:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 27 batchid updated 00000001:00080000:1.0:1787784834.675061:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968141 is committed 00000001:00080000:1.0:1787784834.675068:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968138 is committed 00000001:00080000:1.0:1787784834.675078:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968140 is committed 00000020:00080000:0.0:1787784834.675200:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 28 committed batchid 27 00000020:00080000:0.0:1787784834.675448:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 27 00010000:00010000:0.0:1787784834.993609:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784835.569837:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530507480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668425984:0@lo:400:kworker.0 00000100:00100000:1.0:1787784835.569950:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784835.570068:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668425984 00000100:00100000:2.0:1787784835.570128:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668426112:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784835.570195:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784835.570231:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784835.570250:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305043c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668426240:0@lo:400:kworker.0 00000100:00100000:2.0:1787784835.570282:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784835.570330:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668426112 00000100:00100000:2.0:1787784835.570334:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668426112 00000100:00100000:2.0:1787784835.570394:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1787784835.570407:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668426112:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784835.570451:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784835.570503:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668426112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 100us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784835.570511:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00080000:2.0:1787784835.570538:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784835.570576:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967375 gen 2 00000004:00080000:2.0:1787784835.570583:0:3276:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967375 committed transno 4294967375 00000100:00100000:2.0:1787784835.570627:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530517840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668426112:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784835.571377:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668425984 00000100:00100000:0.0:1787784835.571434:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 580 00000100:00100000:0.0:1787784835.571444:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668425984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784835.572746:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784835.572765:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668426624:0@lo:400:kworker.0 00000100:00100000:2.0:1787784835.572835:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784835.572886:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668426240 00000100:00100000:3.0:1787784835.572895:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668426240 00000100:00100000:3.0:1787784835.572950:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668426624 00000100:00100000:3.0:1787784835.572954:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668426624 00000100:00100000:3.0:1787784835.572997:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1787784835.573007:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668426240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784835.573066:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668426240:12345-0@lo:400:kworker.0 Request processed in 61us (2782us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784835.573074:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1787784835.573091:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:3.0:1787784835.573097:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668426624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784835.573127:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668426624:12345-0@lo:400:kworker.0 Request processed in 30us (292us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784835.573485:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00080000:2.0:1787784835.573750:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784835.573799:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668426624:0@lo:400:kworker.0 00000100:00100000:2.0:1787784835.573852:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305043c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668426240:0@lo:400:kworker.0 00000100:00100000:0.0:1787784835.576384:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668425984:12345-0@lo:400:kworker.0 Request processed in 4406us (5899us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784835.576403:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 580 00000100:00100000:0.0:1787784835.576537:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530506940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668426368:0@lo:400:kworker.0 00000100:00100000:0.0:1787784835.576916:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784835.576973:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784835.576981:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668426496:0@lo:400:kworker.0 00000100:00100000:0.0:1787784835.577017:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784835.577089:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668426368 00000100:00100000:0.0:1787784835.577099:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668426368 00000100:00100000:0.0:1787784835.577191:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1787784835.577201:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668426368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784835.577273:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668426368:12345-0@lo:400:kworker.0 Request processed in 74us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784835.577280:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1787784835.578471:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530506940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668426368:0@lo:400:kworker.0 00000100:00080000:2.0:1787784835.579039:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784835.579369:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668426496 00000100:00100000:3.0:1787784835.579380:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668426496 00000100:00100000:3.0:1787784835.579451:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1787784835.579463:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668426496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784835.579580:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668426496:12345-0@lo:400:kworker.0 Request processed in 120us (2562us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784835.579587:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1787784835.579632:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668426496:0@lo:400:kworker.0 00000100:00100000:1.0:1787784835.581547:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530507480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668425984:0@lo:400:kworker.0 00000100:00080000:2.0:1787784835.582447:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784836.017420:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787784836.271027:0:21520:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 18:53:54 (1787784834) 00000100:00100000:1.0:1787784836.583758:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784836.583878:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206080 00000100:00100000:0.0:1787784836.583893:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206080 00000100:00100000:0.0:1787784836.583965:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 581 00000100:00100000:0.0:1787784836.583978:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:30731:x1874627658206080:12345-192.168.203.43@tcp:101:mount.lustre.0 00010000:00010000:0.0:1787784836.584009:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784836.584033:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989500031000/0xb0ce63137acc6c6b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b13a9 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784836.584070:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989500031000/0xb0ce63137acc6c6b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13a9 expref: 14 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784836.584084:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989500031000/0xb0ce63137acc6c6b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13a9 expref: 14 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784836.584096:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500031000, rc 0) 00000100:00100000:0.0:1787784836.584212:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30731:x1874627658206080:12345-192.168.203.43@tcp:101:mount.lustre.0 Request processed in 237us (470us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.584220:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 581 00000100:00100000:1.0:1787784836.606092:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784836.606205:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206208 00000100:00100000:0.0:1787784836.606219:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206208 00000100:00100000:0.0:1787784836.606306:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 582 00000100:00100000:0.0:1787784836.606319:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30731:x1874627658206208:12345-192.168.203.43@tcp:501:mount.lustre.0 00000100:00100000:0.0:1787784836.606799:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30731:x1874627658206208:12345-192.168.203.43@tcp:501:mount.lustre.0 Request processed in 480us (732us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.606809:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 582 00000100:00100000:1.0:1787784836.609665:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784836.609757:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206336 00000100:00100000:3.0:1787784836.609771:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206336 00000100:00100000:3.0:1787784836.609838:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 583 00000100:00100000:3.0:1787784836.609851:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30731:x1874627658206336:12345-192.168.203.43@tcp:503:mount.lustre.0 00000100:00100000:3.0:1787784836.610560:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30731:x1874627658206336:12345-192.168.203.43@tcp:503:mount.lustre.0 Request processed in 707us (905us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784836.610571:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 583 00000100:00100000:1.0:1787784836.631786:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784836.633078:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206464 00000100:00100000:3.0:1787784836.633098:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206464 00000100:00100000:3.0:1787784836.633372:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00100000:3.0:1787784836.633386:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30735:x1874627658206464:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784836.634802:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30735:x1874627658206464:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 1408us (3020us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784836.634817:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00100000:1.0:1787784836.658069:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784836.658346:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206592 00000100:00100000:3.0:1787784836.658361:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206592 00000100:00100000:3.0:1787784836.658619:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 585 00000100:00100000:3.0:1787784836.658632:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30735:x1874627658206592:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:3.0:1787784836.659077:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30735:x1874627658206592:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 446us (1020us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784836.659087:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 585 00000100:00100000:1.0:1787784836.673768:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784836.674052:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206720 00000100:00100000:2.0:1787784836.674065:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206720 00000100:00100000:2.0:1787784836.674139:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 586 00000100:00100000:2.0:1787784836.674198:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30735:x1874627658206720:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784836.674561:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30735:x1874627658206720:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 368us (807us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784836.674570:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 586 00000100:00100000:1.0:1787784836.678859:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784836.679133:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206848 00000100:00100000:0.0:1787784836.679193:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206848 00000100:00100000:0.0:1787784836.679284:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 587 00000100:00100000:0.0:1787784836.679304:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:30731:x1874627658206848:12345-192.168.203.43@tcp:101:mount.lustre.0 00010000:00010000:0.0:1787784836.679326:0:5803:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784836.679348:0:5803:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff989500030600/0xb0ce63137acc6c72 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b13d3 expref: -99 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784836.679388:0:5803:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff989500030600/0xb0ce63137acc6c72 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13d3 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784836.679401:0:5803:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff989500030600/0xb0ce63137acc6c72 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13d3 expref: 15 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784836.679412:0:5803:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989500030600, rc 0) 00000100:00100000:0.0:1787784836.679491:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+15:30731:x1874627658206848:12345-192.168.203.43@tcp:101:mount.lustre.0 Request processed in 189us (646us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.679498:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 587 00000100:00100000:1.0:1787784836.683981:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784836.684083:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658206976 00000100:00100000:3.0:1787784836.684097:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658206976 00000100:00100000:3.0:1787784836.684192:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00100000:3.0:1787784836.684204:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+15:30731:x1874627658206976:12345-192.168.203.43@tcp:256:mount.lustre.0 20000000:01000000:3.0:1787784836.684257:0:5804:0:(mgs_nids.c:914:mgs_get_ir_logs()) Reading IR log lustre-cliir bufsize 1048576. 20000000:01000000:3.0:1787784836.684289:0:5804:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4052/1/256/255. 20000000:01000000:3.0:1787784836.684304:0:5804:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4008/1/256/255. 20000000:01000000:3.0:1787784836.684308:0:5804:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 3964/1/256/255. 20000000:01000000:3.0:1787784836.684310:0:5804:0:(mgs_nids.c:318:mgs_nidtbl_read()) Read IR logs lustre return with 132, version 5 00000100:00100000:3.0:1787784836.687064:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+15:30731:x1874627658206976:12345-192.168.203.43@tcp:256:mount.lustre.0 Request processed in 2849us (3084us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784836.687081:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00100000:1.0:1787784836.705481:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784836.705620:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658207104 00000100:00100000:2.0:1787784836.705634:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658207104 00000100:00100000:2.0:1787784836.705712:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00100000:2.0:1787784836.705725:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+15:30731:x1874627658207104:12345-192.168.203.43@tcp:101:mount.lustre.0 00010000:00010000:2.0:1787784836.705749:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784836.705774:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff9894022fd800/0xb0ce63137acc6c79 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b13da expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784836.705822:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff9894022fd800/0xb0ce63137acc6c79 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13da expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784836.705835:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff9894022fd800/0xb0ce63137acc6c79 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13da expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784836.705846:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:2.0:1787784836.705919:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30731:x1874627658207104:12345-192.168.203.43@tcp:101:mount.lustre.0 Request processed in 196us (453us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784836.705927:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00100000:1.0:1787784836.714083:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784836.714185:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658207232 00000100:00100000:3.0:1787784836.714198:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658207232 00000100:00100000:3.0:1787784836.714388:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 590 00000100:00100000:3.0:1787784836.714401:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30731:x1874627658207232:12345-192.168.203.43@tcp:501:mount.lustre.0 00000100:00100000:3.0:1787784836.714797:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30731:x1874627658207232:12345-192.168.203.43@tcp:501:mount.lustre.0 Request processed in 397us (727us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784836.714806:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 590 00000100:00100000:1.0:1787784836.720551:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784836.721504:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658207360 00000100:00100000:0.0:1787784836.721522:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658207360 00000100:00100000:0.0:1787784836.721763:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 591 00000100:00100000:0.0:1787784836.721777:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30731:x1874627658207360:12345-192.168.203.43@tcp:503:mount.lustre.0 00000100:00100000:0.0:1787784836.727345:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30731:x1874627658207360:12345-192.168.203.43@tcp:503:mount.lustre.0 Request processed in 3443us (4682us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.727366:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 591 00000100:00100000:1.0:1787784836.735277:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784836.735389:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658207488 00000100:00100000:2.0:1787784836.735403:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658207488 00000100:00100000:2.0:1787784836.735473:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 592 00000100:00100000:2.0:1787784836.735485:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30738:x1874627658207488:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787784836.735949:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30738:x1874627658207488:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 464us (817us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784836.735961:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 592 00000100:00100000:1.0:1787784836.746473:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784836.746592:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658207616 00000100:00100000:0.0:1787784836.746604:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658207616 00000100:00100000:0.0:1787784836.746671:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 593 00000100:00100000:0.0:1787784836.746684:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30738:x1874627658207616:12345-192.168.203.43@tcp:502:llog_process_th.0 00000100:00100000:0.0:1787784836.747027:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:30738:x1874627658207616:12345-192.168.203.43@tcp:502:llog_process_th.0 Request processed in 343us (566us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.747036:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 593 00000100:00100000:1.0:1787784836.767045:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784836.767094:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784836.767125:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784836.767192:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784836.767214:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784836.767552:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208512 00000100:00100000:0.0:1787784836.767565:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208512 00000100:00100000:0.0:1787784836.767650:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208128 00000100:00100000:0.0:1787784836.767654:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208128 00000100:00100000:0.0:1787784836.767682:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:0.0:1787784836.767693:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2355:x1874627658208512:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784836.767796:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2355:x1874627658208512:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 105us (701us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.767805:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:0.0:1787784836.767833:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:0.0:1787784836.767841:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:2355:x1874627658208128:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784836.767864:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:2355:x1874627658208128:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (672us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.767870:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:0.0:1787784836.767917:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208256 00000100:00100000:0.0:1787784836.767921:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208256 00000100:00100000:0.0:1787784836.767944:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208768 00000100:00100000:0.0:1787784836.767947:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208768 00000100:00100000:0.0:1787784836.767962:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208384 00000100:00100000:0.0:1787784836.767965:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208384 00000100:00100000:0.0:1787784836.767981:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 398 00000100:00100000:0.0:1787784836.767987:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658208256:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784836.768016:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658208256:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (988us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.768023:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 398 00000100:00100000:0.0:1787784836.768035:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 399 00000100:00100000:0.0:1787784836.768040:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658208768:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784836.768064:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658208768:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (938us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.768070:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 399 00000100:00100000:0.0:1787784836.768082:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:0.0:1787784836.768088:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2355:x1874627658208384:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784836.768111:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2355:x1874627658208384:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (895us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.768116:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:1.0:1787784836.768861:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784836.768957:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208640 00000100:00100000:2.0:1787784836.768964:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208640 00000100:00100000:2.0:1787784836.769017:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:2.0:1787784836.769027:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658208640:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784836.769073:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658208640:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784836.769081:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00080000:2.0:1787784836.769102:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784836.769107:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784836.769110:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784836.770875:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784836.771012:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658207744 00000100:00100000:3.0:1787784836.771024:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658207744 00000100:00100000:3.0:1787784836.771075:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1258 00000100:00100000:3.0:1787784836.771085:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2353:x1874627658207744:12345-192.168.203.43@tcp:38:mount.lustre.0 00010000:00080000:3.0:1787784836.771121:0:5813:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-MDT0000: connection from 633bab11-ac8b-43e8-9332-f3266553ac37@192.168.203.43@tcp t0 exp 0000000000000000 cur 627 last 0 00000020:00000080:3.0:1787784836.771202:0:5813:0:(genops.c:1385:class_connect()) connect: client 633bab11-ac8b-43e8-9332-f3266553ac37, cookie 0xb0ce63137acc6c80 00000004:00100000:3.0:1787784836.771872:0:5813:0:(mdt_handler.c:7293:mdt_connect_internal()) lustre-MDT0000: cli supports cksum type f7, return 37 00000100:00080000:2.0:1787784836.774098:0:5813:0:(import.c:65:import_set_state_nolock()) ffff98953f0b2000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1787784836.775570:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+6:2353:x1874627658207744:12345-192.168.203.43@tcp:38:mount.lustre.0 Request processed in 4473us (4693us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784836.775594:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1258 00000100:00100000:1.0:1787784836.791910:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784836.792383:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208000 00000100:00100000:0.0:1787784836.792396:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208000 00000100:00100000:0.0:1787784836.792482:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1259 00000100:00100000:0.0:1787784836.792493:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:30731:x1874627658208000:12345-192.168.203.43@tcp:41:mount.lustre.0 00000004:00000024:0.0:1787784836.792548:0:5811:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 18087936 pending 0 free 2259681280 avail 2257584128 00000100:00100000:0.0:1787784836.792646:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:30731:x1874627658208000:12345-192.168.203.43@tcp:41:mount.lustre.0 Request processed in 155us (750us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.792654:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1259 00000100:00100000:1.0:1787784836.792971:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784836.793003:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658207872 00000100:00100000:1.0:1787784836.793008:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658207872 00000100:00100000:1.0:1787784836.793039:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1260 00000100:00100000:1.0:1787784836.793048:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+6:0:x1874627658207872:12345-192.168.203.43@tcp:41:mount.lustre.0 00000004:00000024:1.0:1787784836.793063:0:10850:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 18087936 pending 0 free 2259681280 avail 2257584128 00000100:00100000:1.0:1787784836.793093:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+6:0:x1874627658207872:12345-192.168.203.43@tcp:41:mount.lustre.0 Request processed in 47us (123us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784836.793099:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1260 00000100:00080000:2.0:1787784836.793309:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784836.800394:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784836.800804:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658208896 00000100:00100000:2.0:1787784836.800816:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658208896 00000100:00100000:2.0:1787784836.800887:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1261 00000100:00100000:2.0:1787784836.800899:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+6:30731:x1874627658208896:12345-192.168.203.43@tcp:40:mount.lustre.0 00000100:00100000:2.0:1787784836.801020:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+6:30731:x1874627658208896:12345-192.168.203.43@tcp:40:mount.lustre.0 Request processed in 123us (640us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784836.801029:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1261 00000100:00100000:2.0:1787784836.805980:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784836.806028:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784836.806122:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784836.806263:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658209280 00000100:00100000:3.0:1787784836.806277:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658209280 00000100:00100000:3.0:1787784836.806347:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 256 00000100:00100000:3.0:1787784836.806359:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+6:30731:x1874627658209280:12345-192.168.203.43@tcp:33:mount.lustre.0 00000004:00200000:3.0:1787784836.806760:0:5815:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000100:00100000:3.0:1787784836.806932:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+6:30731:x1874627658209280:12345-192.168.203.43@tcp:33:mount.lustre.0 Request processed in 573us (966us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784836.806943:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 256 00000100:00100000:1.0:1787784836.808682:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658209152 00000100:00100000:0.0:1787784836.808682:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658209024 00000100:00100000:0.0:1787784836.808701:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658209024 00000100:00100000:1.0:1787784836.808702:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658209152 00000100:00100000:1.0:1787784836.808774:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 348 00000100:00100000:0.0:1787784836.808793:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:1.0:1787784836.808795:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0+-99:2353:x1874627658209152:12345-192.168.203.43@tcp:8:mount.lustre.0 00000100:00100000:0.0:1787784836.808804:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2353:x1874627658209024:12345-192.168.203.43@tcp:8:mount.lustre.0 00010000:00080000:1.0:1787784836.808841:0:6671:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from 633bab11-ac8b-43e8-9332-f3266553ac37@192.168.203.43@tcp t0 exp 0000000000000000 cur 627 last 0 00010000:00080000:0.0:1787784836.808841:0:6670:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from 633bab11-ac8b-43e8-9332-f3266553ac37@192.168.203.43@tcp t0 exp 0000000000000000 cur 627 last 0 00000020:00000080:1.0:1787784836.808902:0:6671:0:(genops.c:1385:class_connect()) connect: client 633bab11-ac8b-43e8-9332-f3266553ac37, cookie 0xb0ce63137acc6c8e 00000020:00000080:0.0:1787784836.808903:0:6670:0:(genops.c:1385:class_connect()) connect: client 633bab11-ac8b-43e8-9332-f3266553ac37, cookie 0xb0ce63137acc6c87 00002000:00100000:0.0:1787784836.808919:0:6670:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli 633bab11-ac8b-43e8-9332-f3266553ac37/ffff989540500000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113936 ocd_grant: 134217728 ocd_index: 0 ocd_group 0 00002000:00100000:1.0:1787784836.808923:0:6671:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 633bab11-ac8b-43e8-9332-f3266553ac37/ffff98953aec9800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113936 ocd_grant: 134217728 ocd_index: 0 ocd_group 0 00002000:00100000:1.0:1787784836.808936:0:6671:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return 37 00002000:00100000:0.0:1787784836.808949:0:6670:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return 37 00002000:00080000:0.0:1787784836.809199:0:6670:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00002000:00080000:1.0:1787784836.809204:0:6671:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:1.0:1787784836.809260:0:6671:0:(import.c:65:import_set_state_nolock()) ffff98953aecc800 : changing import state from RECOVER to FULL 00000100:00080000:0.0:1787784836.809260:0:6670:0:(import.c:65:import_set_state_nolock()) ffff989540502800 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1787784836.809343:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+6:2353:x1874627658209152:12345-192.168.203.43@tcp:8:mount.lustre.0 Request processed in 547us (3216us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.809344:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+6:2353:x1874627658209024:12345-192.168.203.43@tcp:8:mount.lustre.0 Request processed in 538us (3310us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784836.809355:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 349 00000100:00100000:1.0:1787784836.809366:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 348 00010000:00010000:0.0:1787784837.043930:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784837.167729:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784837.167811:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658209408 00000100:00100000:1.0:1787784837.167829:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658209408 00000100:00100000:1.0:1787784837.168018:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1262 00000100:00100000:1.0:1787784837.168033:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+6:30779:x1874627658209408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784837.168070:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784837.168099:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6c95 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1404 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.168386:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.168405:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.168419:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784837.168689:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784837.168695:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 30 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784837.168782:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.168815:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6c95 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 8 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.168837:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 7 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.168847:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:1.0:1787784837.168850:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784837.168855:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1cc00, rc 0) 00000100:00100000:1.0:1787784837.168955:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+7:30779:x1874627658209408:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 926us (1242us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784837.168965:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1262 00000100:00100000:2.0:1787784837.187773:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784837.187924:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658209536 00000100:00100000:3.0:1787784837.187936:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658209536 00000100:00100000:3.0:1787784837.188030:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1263 00000100:00100000:3.0:1787784837.188040:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+7:30780:x1874627658209536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784837.188065:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784837.188089:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6ca3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b140b expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188200:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188233:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188247:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188263:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188271:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188279:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188786:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6cb8 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.188798:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6cb8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.189196:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6cb8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784837.189322:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x1:0x0]: returning size 6656 00000004:00200000:3.0:1787784837.189326:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784837.190998:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6cb8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191026:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191033:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191042:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191049:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784837.191054:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191063:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191081:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:3.0:1787784837.191084:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784837.191088:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191094:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6caa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191114:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191119:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191124:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191128:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784837.191130:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191135:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191140:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b979c7a (ffff9895198b5900) 00010000:00010000:3.0:1787784837.191142:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784837.191144:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.191201:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6cb1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.192917:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6ca3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b140b expref: 9 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.192956:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6cb8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b140b expref: 8 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.192973:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:3.0:1787784837.192977:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784837.192981:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784837.193099:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+8:30780:x1874627658209536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5058us (5336us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784837.193111:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1263 00000100:00100000:2.0:1787784837.199006:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784837.201862:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658209792 00000100:00100000:0.0:1787784837.201883:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658209792 00000100:00100000:0.0:1787784837.201979:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1264 00000100:00100000:0.0:1787784837.201990:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+8:30780:x1874627658209792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784837.202018:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784837.202043:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc6cbf lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1412 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202136:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202181:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202196:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202234:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202242:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202249:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202862:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc6cd4 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202876:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc6cd4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.202883:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc6cd4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784837.202893:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:0.0:1787784837.203095:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203103:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203110:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203116:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784837.203122:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203135:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203181:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:0.0:1787784837.203185:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784837.203191:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203197:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cc6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203213:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203219:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203236:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203241:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784837.203244:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203249:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203256:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff989520108700) 00010000:00010000:0.0:1787784837.203259:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784837.203262:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203272:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ccd lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203772:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc6cbf lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1412 expref: 11 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203792:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc6cd4 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1412 expref: 10 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.203800:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784837.203803:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784837.203806:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b800, rc 0) 00000100:00100000:0.0:1787784837.203905:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:30780:x1874627658209792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1916us (4913us total) trans 4294968142 rc 0/0 00000100:00100000:0.0:1787784837.203915:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1264 00000100:00100000:1.0:1787784837.210626:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.210784:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658209920 00000100:00100000:2.0:1787784837.210797:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658209920 00000100:00100000:2.0:1787784837.210886:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1265 00000100:00100000:2.0:1787784837.210899:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+10:30780:x1874627658209920:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1787784837.211039:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+10:30780:x1874627658209920:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 143us (427us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784837.211048:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1265 00000100:00100000:3.0:1787784837.234487:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784837.239860:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658210048 00000100:00100000:0.0:1787784837.239882:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658210048 00000100:00100000:0.0:1787784837.240000:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1266 00000100:00100000:0.0:1787784837.240013:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30781:x1874627658210048:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784837.240041:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784837.240069:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc6ce2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1419 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.240173:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.240191:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.240205:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.240722:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.240735:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.240744:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.245995:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246018:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246028:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246035:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.246042:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246051:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246073:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:1.0:1787784837.246077:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.246083:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246091:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6ce9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246111:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246123:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246129:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246134:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.246137:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246144:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246203:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff989520108700) 00010000:00010000:1.0:1787784837.246206:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.246209:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246215:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6cf0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246666:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc6ce2 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1419 expref: 98 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246680:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:1.0:1787784837.246683:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.246687:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc6ce2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1419 expref: 98 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.246702:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150be00, rc 0) 00000100:00100000:1.0:1787784837.246983:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30781:x1874627658210048:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6796us (12336us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784837.246994:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1266 00000100:00100000:1.0:1787784837.252025:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.252116:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658210304 00000100:00100000:2.0:1787784837.252127:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658210304 00000100:00100000:2.0:1787784837.252246:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1267 00000100:00100000:2.0:1787784837.252258:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30781:x1874627658210304:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:2.0:1787784837.252331:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784837.252375:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc685f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1252 expref: 97 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.252586:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:2.0:1787784837.252592:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784837.253626:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.253651:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.253659:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 9 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.253667:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.253710:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 9 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.253728:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 9 pid: 10850 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784837.253743:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668426752:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784837.253888:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 44 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.253897:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 44 pid: 5813 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784837.253906:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035da940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668426880:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784837.253930:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5540 going to sleep for 11 seconds 00000100:00100000:2.0:1787784837.259043:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668426752:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784837.259065:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035da940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668426880:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784837.259109:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784837.264783:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784837.265524:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784837.265849:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658210432 00000100:00100000:0.0:1787784837.265863:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658210432 00010000:00010000:0.0:1787784837.265919:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 10 pid: 10850 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784837.265971:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 379 00000100:00100000:0.0:1787784837.265983:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:633bab11-ac8b-43e8-9332-f3266553ac37+10:2357:x1874627658210432:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784837.266001:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784837.266009:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 10 pid: 10850 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.266025:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 10 pid: 10850 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.266055:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6c9c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1404 expref: 10 pid: 10850 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.266082:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:0.0:1787784837.266088:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.266099:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784837.266201:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:633bab11-ac8b-43e8-9332-f3266553ac37+9:2357:x1874627658210432:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 220us (1433us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784837.266209:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 379 00000100:00100000:3.0:1787784837.267087:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658210560 00000100:00100000:3.0:1787784837.267096:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658210560 00010000:00010000:3.0:1787784837.267129:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 45 pid: 5813 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784837.267196:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 380 00000100:00100000:3.0:1787784837.267206:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2357:x1874627658210560:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784837.267658:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784837.267670:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 45 pid: 5813 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.267686:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 45 pid: 5813 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.267709:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc689e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1267 expref: 45 pid: 5813 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.267728:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:3.0:1787784837.267735:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.267746:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.267755:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.267794:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784837.267847:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:2357:x1874627658210560:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 642us (2326us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784837.267855:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 380 00010000:00010000:1.0:1787784837.268331:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.268342:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.268817:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.268843:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.268854:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784837.272995:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 512 00010000:00010000:1.0:1787784837.273021:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.273034:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.273043:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.273052:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.273058:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.273067:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.273087:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5600) 00010000:00010000:1.0:1787784837.273091:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.273097:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.273103:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6cf7 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784837.273127:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001298cf41 reply state 00000000173bc462 transno 4294968143 ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784837.273137:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784837.273310:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30781:x1874627658210304:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 21047us (21291us total) trans 4294968143 rc 0/0 00000100:00100000:1.0:1787784837.273324:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1267 00010000:00010000:1.0:1787784837.281554:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.281578:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.281586:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.281599:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.281604:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.281613:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.281636:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff98950a38cf00) 00010000:00010000:1.0:1787784837.281639:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.281644:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.281651:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6cfe lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784837.309434:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784837.309542:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658210688 00000100:00100000:3.0:1787784837.309555:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658210688 00000100:00100000:3.0:1787784837.309631:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1268 00000100:00100000:3.0:1787784837.309643:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+44:30785:x1874627658210688:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:3.0:1787784837.309681:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784837.309710:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6d05 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1420 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.309795:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6d0c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.309820:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6d0c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.309835:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6d0c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784837.309915:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784837.309920:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 31 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784837.309951:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6d0c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.309982:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6d05 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1420 expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.310005:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6d0c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1420 expref: 45 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.310015:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5900) 00010000:00010000:3.0:1787784837.310019:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784837.310023:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784837.310109:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30785:x1874627658210688:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 467us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784837.310116:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1268 00000100:00100000:3.0:1787784837.314472:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784837.314525:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.314636:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658211072 00000100:00100000:0.0:1787784837.314636:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658210816 00000100:00100000:0.0:1787784837.314648:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658210816 00000100:00100000:2.0:1787784837.314649:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658211072 00000100:00100000:2.0:1787784837.314914:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1269 00000100:00100000:2.0:1787784837.314927:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30785:x1874627658211072:12345-192.168.203.43@tcp:36:mkdir.0 00010000:00010000:2.0:1787784837.314998:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784837.315030:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6d0c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1420 expref: 46 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.315054:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5900) 00010000:00010000:2.0:1787784837.315066:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784837.316324:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1270 00000100:00100000:0.0:1787784837.316345:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2354:x1874627658210816:12345-192.168.203.43@tcp:41:mkdir.0 00000004:00000024:0.0:1787784837.316395:0:10850:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 18087936 pending 0 free 2259681280 avail 2257584128 00000100:00100000:0.0:1787784837.316496:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+45:2354:x1874627658210816:12345-192.168.203.43@tcp:41:mkdir.0 Request processed in 158us (2037us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784837.316505:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1270 00000100:00100000:2.0:1787784837.317271:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+44:30785:x1874627658211072:12345-192.168.203.43@tcp:36:mkdir.0 Request processed in 2343us (2741us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787784837.317281:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1269 00000100:00100000:0.0:1787784837.320346:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.320452:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658211200 00000100:00100000:2.0:1787784837.320466:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658211200 00000100:00100000:2.0:1787784837.320577:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1271 00000100:00100000:2.0:1787784837.320589:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+44:30785:x1874627658211200:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:2.0:1787784837.320617:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784837.320648:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d13 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1427 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.320928:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.320947:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.320961:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784837.321034:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784837.321039:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 31 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784837.321064:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.321092:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d13 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.321112:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.321122:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784837.321125:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784837.321129:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:2.0:1787784837.321366:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30785:x1874627658211200:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 778us (1034us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784837.321375:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1271 00000100:00100000:3.0:1787784837.323916:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.324180:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658211328 00000100:00100000:2.0:1787784837.324193:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658211328 00000100:00100000:2.0:1787784837.324278:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1272 00000100:00100000:2.0:1787784837.324290:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+45:30785:x1874627658211328:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:2.0:1787784837.324316:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784837.324346:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc6d21 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b142e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324430:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324446:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324459:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324480:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324488:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324500:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324628:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6d36 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324638:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6d36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324646:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6d36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784837.324706:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 512 00000004:00200000:2.0:1787784837.324711:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784837.324739:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6d36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324754:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324761:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324768:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324776:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989508905000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.324781:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324790:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324817:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784837.324821:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.324827:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324834:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6d28 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324849:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324856:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324862:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324868:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.324871:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324877:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324886:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff989520131e00) 00010000:00010000:2.0:1787784837.324888:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.324892:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324898:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6d2f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324939:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc6d21 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b142e expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324956:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6d36 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b142e expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.324964:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff989520131000) 00010000:00010000:2.0:1787784837.324967:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784837.324970:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:2.0:1787784837.325588:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30785:x1874627658211328:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 1295us (1682us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784837.325599:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1272 00000100:00100000:3.0:1787784837.333410:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784837.333544:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658211456 00000100:00100000:1.0:1787784837.333557:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658211456 00000100:00100000:1.0:1787784837.333685:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1273 00000100:00100000:1.0:1787784837.333698:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30786:x1874627658211456:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784837.333725:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784837.333751:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6d3d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1435 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.333993:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334012:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334031:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334048:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334054:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334061:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334188:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334197:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334204:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334211:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.334295:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334306:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334321:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff989520131000) 00010000:00010000:1.0:1787784837.334325:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.334330:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334337:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d44 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334352:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334358:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334364:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334370:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1cc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.334373:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334379:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334387:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff98950a38cf00) 00010000:00010000:1.0:1787784837.334389:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.334392:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334398:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334438:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6d3d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1435 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334456:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff989520131000) 00010000:00010000:1.0:1787784837.334458:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.334462:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6d3d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1435 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.334473:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1c000, rc 0) 00000100:00100000:1.0:1787784837.334546:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30786:x1874627658211456:12345-192.168.203.43@tcp:101:rm.0 Request processed in 850us (1150us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784837.334555:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1273 00000100:00100000:1.0:1787784837.345035:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.345128:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658211712 00000100:00100000:2.0:1787784837.345142:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658211712 00000100:00100000:2.0:1787784837.345279:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1274 00000100:00100000:2.0:1787784837.345292:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30787:x1874627658211712:12345-192.168.203.43@tcp:101:dd.0 00010000:00010000:2.0:1787784837.345321:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784837.345325:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787784837.345360:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6d36 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b142e expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.345386:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff989520131000) 00010000:00010000:2.0:1787784837.345400:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784837.345418:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6d52 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b143c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.345594:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.345610:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.345622:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.345636:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.345642:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.345648:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784837.346515:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x19:0x0], index 0: 0 00010000:00010000:2.0:1787784837.346714:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346729:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346740:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346776:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346789:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346796:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346802:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.346807:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346814:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346824:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784837.346828:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.346832:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346839:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6d59 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784837.346853:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ed825b8c reply state 00000000f239211c transno 4294968144 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784837.346862:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784837.346896:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6d52 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346914:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.346928:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:2.0:1787784837.346931:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784837.346933:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166e800, rc 0) 00000100:00100000:2.0:1787784837.347017:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30787:x1874627658211712:12345-192.168.203.43@tcp:101:dd.0 Request processed in 1727us (1998us total) trans 4294968144 rc 0/0 00000100:00100000:2.0:1787784837.347026:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1274 00010000:00010000:2.0:1787784837.354609:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.354634:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.354644:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.354652:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.354657:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.354667:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.354701:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff989520131200) 00010000:00010000:2.0:1787784837.354704:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.354710:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.354718:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6d60 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784837.380087:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784837.380353:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658211840 00000100:00100000:0.0:1787784837.380366:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658211840 00000100:00100000:0.0:1787784837.380438:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 350 00000100:00100000:0.0:1787784837.380450:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:30787:x1874627658211840:12345-192.168.203.43@tcp:101:dd.0 00010000:00010000:0.0:1787784837.380476:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784837.380505:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1443 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784837.380547:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784837.380575:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=628s v=(0 0) 00010000:00010000:0.0:1787784837.380582:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784837.380595:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.380613:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784837.380629:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784837.380710:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:30787:x1874627658211840:12345-192.168.203.43@tcp:101:dd.0 Request processed in 262us (640us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784837.380719:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 350 00000100:00100000:0.0:1787784837.397494:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.397637:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658211968 00000100:00100000:2.0:1787784837.397651:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658211968 00002000:00100000:2.0:1787784837.397727:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff989402add2c0 x1874627658211968/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:718/0 lens 488/0 e 0 to 0 dl 1787784848 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784837.397740:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402add2c0 with x1874627658211968 ext(0->1048575) 00002000:00010000:2.0:1787784837.397747:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402add2c0 00000100:00100000:2.0:1787784837.397826:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 111 00000100:00100000:2.0:1787784837.397837:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658211968:12345-192.168.203.43@tcp:4:dd.0 00000100:00100000:3.0:1787784837.462603:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658211968:12345-192.168.203.43@tcp:4:dd.0 Request processed in 64758us (65117us total) trans 4294967365 rc 0/0 00000100:00100000:3.0:1787784837.462617:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 111 00002000:00100000:3.0:1787784837.462629:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff989402add2c0 x1874627658211968/t4294967365(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:718/0 lens 488/448 e 0 to 0 dl 1787784848 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784837.462641:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402add2c0 with x1874627658211968 ext(0->1048575) 00002000:00010000:3.0:1787784837.462647:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989402add2c0 00000100:00100000:0.0:1787784837.469564:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.469680:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658212096 00000100:00100000:2.0:1787784837.469693:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658212096 00000100:00100000:2.0:1787784837.469835:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1275 00000100:00100000:2.0:1787784837.469858:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30787:x1874627658212096:12345-192.168.203.43@tcp:44:dd.0 00000040:00080000:1.0:1787784837.554050:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784837.554065:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784837.554903:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:1.0:1787784837.554919:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784837.570392:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30787:x1874627658212096:12345-192.168.203.43@tcp:44:dd.0 Request processed in 100529us (100831us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784837.570409:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1275 00000001:00080000:1.0:1787784837.570550:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968142 is committed 00000020:00080000:1.0:1787784837.570568:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 28 batchid updated 00000001:00080000:1.0:1787784837.570590:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968143 is committed 00000001:00080000:1.0:1787784837.570597:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968144 is committed 00000001:00100000:1.0:1787784837.570603:0:5833:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 633bab11-ac8b-43e8-9332-f3266553ac37 00000020:00080000:3.0:1787784837.570882:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 28 00000020:00080000:3.0:1787784837.570895:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 28 00000100:00100000:3.0:1787784837.573679:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.573855:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658212224 00000100:00100000:2.0:1787784837.573869:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658212224 00000100:00100000:2.0:1787784837.573946:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 351 00000100:00100000:2.0:1787784837.573957:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658212224:12345-192.168.203.43@tcp:16:dd.0 00000001:00100000:1.0:1787784837.701037:0:6696:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 633bab11-ac8b-43e8-9332-f3266553ac37 00000001:00080000:1.0:1787784837.701076:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967365 is committed 00000100:00100000:2.0:1787784837.701512:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658212224:12345-192.168.203.43@tcp:16:dd.0 Request processed in 127547us (127837us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784837.701528:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 351 00000100:00100000:2.0:1787784837.706688:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784837.708820:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658212480 00000100:00100000:3.0:1787784837.708842:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658212480 00000100:00100000:3.0:1787784837.708931:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 257 00000100:00100000:3.0:1787784837.708943:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30787:x1874627658212480:12345-192.168.203.43@tcp:35:dd.0 00000100:00100000:3.0:1787784837.709611:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30787:x1874627658212480:12345-192.168.203.43@tcp:35:dd.0 Request processed in 667us (2943us total) trans 4294968145 rc 0/0 00000100:00100000:3.0:1787784837.709621:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 257 00000100:00100000:0.0:1787784837.744182:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784837.744709:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658212608 00000100:00100000:1.0:1787784837.744723:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658212608 00000100:00100000:1.0:1787784837.744801:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1276 00000100:00100000:1.0:1787784837.744813:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30788:x1874627658212608:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784837.744847:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784837.744874:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d7c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b144a expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.744957:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.744975:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.744989:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745006:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745013:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745020:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745570:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6d91 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745583:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6d91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745591:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6d91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784837.745698:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 7680 00000004:00200000:1.0:1787784837.745704:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784837.745783:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6d91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745798:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745805:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745812:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745820:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.745825:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745833:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745846:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:1.0:1787784837.745849:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.745855:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745862:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6d83 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745874:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745879:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745890:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745895:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1da00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784837.745898:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745904:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745910:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff98950a38c300) 00010000:00010000:1.0:1787784837.745913:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784837.745915:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745920:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6d8a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745956:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6d7c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b144a expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745970:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1d800/0xb0ce63137acc6d91 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b144a expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.745978:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:1.0:1787784837.745980:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784837.745983:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1d800, rc 0) 00000100:00100000:1.0:1787784837.746070:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:30788:x1874627658212608:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1259us (1931us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784837.746083:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1276 00000100:00100000:0.0:1787784837.803722:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.803852:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658212736 00000100:00100000:2.0:1787784837.803865:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658212736 00000100:00100000:2.0:1787784837.803941:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1277 00000100:00100000:2.0:1787784837.803953:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+96:30790:x1874627658212736:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:2.0:1787784837.803986:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784837.804012:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc6d98 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1451 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.804098:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc6d9f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.804116:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc6d9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.804128:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc6d9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784837.804456:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784837.804462:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 31 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784837.804573:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc6d9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.804609:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc6d98 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1451 expref: 98 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.804631:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc6d9f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1451 expref: 97 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.804642:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784837.804645:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784837.804649:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166e000, rc 0) 00000100:00100000:2.0:1787784837.804742:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30790:x1874627658212736:12345-192.168.203.43@tcp:101:stat.0 Request processed in 791us (1035us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784837.804751:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1277 00000100:00100000:2.0:1787784837.811133:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784837.811697:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658212864 00000100:00100000:3.0:1787784837.811712:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658212864 00000100:00100000:3.0:1787784837.811789:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1278 00000100:00100000:3.0:1787784837.811802:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+97:30790:x1874627658212864:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:3.0:1787784837.811832:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784837.811859:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6da6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1458 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.812134:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.812192:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.812210:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.820127:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.820488:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.820508:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.823747:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc6dbb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.823773:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc6dbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.823789:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc6dbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784837.823894:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 7680 00000004:00200000:2.0:1787784837.823899:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784837.824005:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc6dbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824024:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824032:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824040:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824057:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.824064:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824073:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824090:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784837.824094:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.824100:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824107:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6dad lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824123:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824131:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824138:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824144:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166ec00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.824204:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824210:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824910:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff989520131200) 00010000:00010000:2.0:1787784837.824914:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.824919:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.824926:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6db4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.825035:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6da6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1458 expref: 99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.825104:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ea00/0xb0ce63137acc6dbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1458 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.825113:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:2.0:1787784837.825116:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784837.825119:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166ea00, rc 0) 00000100:00100000:2.0:1787784837.825671:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30790:x1874627658212864:12345-192.168.203.43@tcp:101:stat.0 Request processed in 13867us (14550us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784837.825682:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1278 00000100:00100000:0.0:1787784837.840349:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.841085:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658212992 00000100:00100000:2.0:1787784837.841100:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658212992 00000100:00100000:2.0:1787784837.841192:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1279 00000100:00100000:2.0:1787784837.841204:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+98:30790:x1874627658212992:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:2.0:1787784837.841249:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784837.841278:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6dc2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b145f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841362:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841394:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841407:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841424:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841432:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841439:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841866:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841876:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841884:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784837.841928:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:2.0:1787784837.841933:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784837.841968:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841982:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.841995:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842002:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842010:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842016:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166f800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.842021:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842029:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842042:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:2.0:1787784837.842045:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.842050:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842058:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6dc9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842073:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842079:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842086:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842092:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784837.842095:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842102:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842109:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff989520131200) 00010000:00010000:2.0:1787784837.842112:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784837.842115:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842122:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6dd0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842228:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6dc2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842245:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.842253:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:2.0:1787784837.842256:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784837.842259:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2600, rc 0) 00000100:00100000:2.0:1787784837.842338:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30790:x1874627658212992:12345-192.168.203.43@tcp:101:stat.0 Request processed in 1135us (2003us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784837.842347:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1279 00000100:00100000:0.0:1787784837.848455:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784837.848579:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658213120 00000100:00100000:1.0:1787784837.848592:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658213120 00000100:00100000:1.0:1787784837.848677:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 352 00000100:00100000:1.0:1787784837.848690:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658213120:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:1.0:1787784837.848716:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784837.848740:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6dde lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1466 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784837.848818:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668427008:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784837.848952:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503fa0cc0 going to sleep for 11 seconds 00002000:00010000:2.0:1787784837.857269:0:6671:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:2.0:1787784837.857284:0:6671:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb mtime: 0 -> 1787784836 00002000:00010000:2.0:1787784837.857288:0:6671:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb atime: 0 -> 1787784836 00002000:00010000:2.0:1787784837.857290:0:6671:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb ctime: 0 -> 1787784836 00002000:00010000:2.0:1787784837.857292:0:6671:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:2.0:1787784837.857326:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668427008:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784837.857398:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6dde lrc: 1/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1466 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.857429:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4594048/0xb0ce63137acc6dde lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1466 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.857445:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:2.0:1787784837.857761:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658213120:12345-192.168.203.43@tcp:101:stat.0 Request processed in 9073us (9318us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784837.857772:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 352 00000100:00100000:3.0:1787784837.875023:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784837.875194:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658213248 00000100:00100000:0.0:1787784837.875208:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658213248 00000100:00100000:0.0:1787784837.875307:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 353 00000100:00100000:0.0:1787784837.875320:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658213248:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784837.875345:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784837.875368:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc6de5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b146d expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784837.875445:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adde00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668427136:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784837.875561:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784837.891525:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adde00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668427136:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784837.891582:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc6de5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b146d expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.891612:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc6de5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b146d expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784837.891628:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150be00, rc 0) 00000100:00100000:3.0:1787784837.891704:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658213248:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 16385us (16694us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784837.891717:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 353 00000100:00100000:2.0:1787784837.898483:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784837.898747:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658213376 00000100:00100000:1.0:1787784837.898760:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658213376 00000100:00100000:1.0:1787784837.898836:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1280 00000100:00100000:1.0:1787784837.898848:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30791:x1874627658213376:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784837.898886:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784837.898914:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6dec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1474 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.898999:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.899015:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.899028:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784837.899344:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 7680 00000004:00200000:1.0:1787784837.899349:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784837.899429:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.899468:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6dec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 101 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.899488:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 100 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784837.899498:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:1.0:1787784837.899501:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784837.899505:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404e1da00, rc 0) 00000100:00100000:1.0:1787784837.899587:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30791:x1874627658213376:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 742us (1119us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784837.899596:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1280 00000100:00100000:0.0:1787784837.905586:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784837.906346:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658213504 00000100:00100000:2.0:1787784837.906361:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658213504 00000100:00100000:2.0:1787784837.906438:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 354 00000100:00100000:2.0:1787784837.906449:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658213504:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784837.906472:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784837.906493:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6dfa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b147b expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784837.906568:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668427264:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784837.906825:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506773c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787784837.911996:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668427264:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784837.912060:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6dfa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b147b expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.912092:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6dfa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b147b expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784837.912108:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:2.0:1787784837.912255:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658213504:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 5807us (6681us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784837.912265:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 354 00000100:00100000:1.0:1787784837.924267:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784837.924425:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658213760 00000100:00100000:3.0:1787784837.924439:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658213760 00000100:00100000:3.0:1787784837.924518:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1281 00000100:00100000:3.0:1787784837.924531:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30791:x1874627658213760:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784837.925374:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].37, 136 off13224 00000100:00100000:3.0:1787784837.925486:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30791:x1874627658213760:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 955us (1231us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784837.925496:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1281 00010000:00010000:3.0:1787784838.065408:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784838.578016:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784838.578029:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784838.578073:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 38 count 5 00000040:00080000:2.0:1787784838.578086:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784838.578826:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45e580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668427392:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784838.578964:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895012549c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784838.588825:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45e580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668427392:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784838.592182:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784838.592316:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658213888 00000100:00100000:2.0:1787784838.592328:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658213888 00000100:00100000:2.0:1787784838.592420:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 3 00000100:00100000:2.0:1787784838.592434:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:633bab11-ac8b-43e8-9332-f3266553ac37+7:30801:x1874627658213888:12345-192.168.203.43@tcp:900:lhsmtool_posix.0 00000100:00100000:2.0:1787784838.592546:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:633bab11-ac8b-43e8-9332-f3266553ac37+7:30801:x1874627658213888:12345-192.168.203.43@tcp:900:lhsmtool_posix.0 Request processed in 116us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784838.592555:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 3 00000100:00080000:1.0:1787784838.593058:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784838.598016:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784838.598119:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658214016 00000100:00100000:2.0:1787784838.598132:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658214016 00000100:00100000:2.0:1787784838.598248:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1282 00000100:00100000:2.0:1787784838.598261:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+7:30801:x1874627658214016:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784838.598305:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784838.598459:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+7:30801:x1874627658214016:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 191us (444us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784838.598468:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1282 00000100:00100000:2.0:1787784838.608395:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784838.609040:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658214272 00000100:00100000:3.0:1787784838.609055:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658214272 00000100:00100000:3.0:1787784838.609131:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1283 00000100:00100000:3.0:1787784838.609143:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+7:30801:x1874627658214272:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784838.609195:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784838.609238:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6e01 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1482 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.609368:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.611338:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.611357:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784838.611392:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:2.0:1787784838.611669:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc6e01 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 10 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.611694:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 9 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.611705:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:2.0:1787784838.611708:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784838.611712:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:2.0:1787784838.611814:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+9:30801:x1874627658214272:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2672us (3435us total) trans 4294968146 rc 0/0 00000100:00100000:2.0:1787784838.611824:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1283 00000100:00100000:0.0:1787784838.622937:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784838.623712:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658214400 00000100:00100000:2.0:1787784838.623727:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658214400 00000100:00100000:2.0:1787784838.623834:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 355 00000100:00100000:2.0:1787784838.623847:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2356:x1874627658214400:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784838.623906:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787784838.623907:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784838.623939:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 8 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.623949:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x19:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784838.623954:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784838.623959:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.624001:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 8 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.624018:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 8 pid: 13825 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784838.624035:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45c780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668427520:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784838.624140:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98951084e780 going to sleep for 11 seconds 00000100:00100000:1.0:1787784838.642890:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784838.642928:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658214528 00000100:00100000:1.0:1787784838.642938:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658214528 00010000:00010000:1.0:1787784838.643002:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 9 pid: 13825 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784838.643066:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 381 00000100:00100000:1.0:1787784838.643077:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658214528:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784838.643096:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784838.643126:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 9 pid: 13825 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784838.643143:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 9 pid: 13825 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784838.643234:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784838.643240:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784838.643247:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784838.643259:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784838.643288:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784838.643363:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658214528:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 287us (486us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784838.643372:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 381 00000100:00080000:1.0:1787784838.643410:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787784838.644513:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f87905ca/0xb0ce63137acc6d75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1443 expref: 8 pid: 13825 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784838.644741:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45c780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668427520:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784838.644769:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644782:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644790:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644831:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644840:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644851:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644859:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784838.644864:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644874:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644893:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.644900:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000164fa6ca/0xb0ce63137acc6e16 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784838.644995:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2356:x1874627658214400:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 21150us (22072us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784838.645008:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 355 00000100:00100000:0.0:1787784838.650889:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784838.650937:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658214784 00000100:00100000:0.0:1787784838.650949:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658214784 00000100:00100000:0.0:1787784838.651012:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1284 00000100:00100000:0.0:1787784838.651023:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+9:30801:x1874627658214784:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784838.651177:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+9:30801:x1874627658214784:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 154us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784838.651186:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1284 00000100:00100000:0.0:1787784838.651435:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adde00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668427648:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784838.651483:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784838.651535:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668427648 00000100:00100000:0.0:1787784838.651540:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668427648 00000100:00100000:0.0:1787784838.651586:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1787784838.651593:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668427648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784838.651626:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1787784838.651668:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668427648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784838.651674:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00080000:0.0:1787784838.651694:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784838.651737:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adde00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668427648:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784838.656304:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784838.656386:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658214912 00000100:00100000:3.0:1787784838.656398:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658214912 00000100:00100000:3.0:1787784838.656473:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 356 00000100:00100000:3.0:1787784838.656486:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:633bab11-ac8b-43e8-9332-f3266553ac37+5:2356:x1874627658214912:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784838.656510:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784838.656536:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1489 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784838.656569:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 1048576, mtime: 0x6a8f6e84, atime: 0x0, ctime: 0x6a8f6e84, blocks: 0x803 00010000:00010000:3.0:1787784838.656604:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=629s v=(0 0) 00010000:00010000:3.0:1787784838.656611:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784838.656622:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 6 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.656643:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 6 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.656669:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784838.656742:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:633bab11-ac8b-43e8-9332-f3266553ac37+6:2356:x1874627658214912:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 259us (450us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784838.656751:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 356 00000100:00100000:1.0:1787784838.665895:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784838.665944:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658215168 00000100:00100000:1.0:1787784838.665957:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658215168 00000100:00100000:1.0:1787784838.666028:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1285 00000100:00100000:1.0:1787784838.666040:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+9:30801:x1874627658215168:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784838.666205:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+9:30801:x1874627658215168:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 166us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784838.666226:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1285 00000100:00100000:3.0:1787784838.680072:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784838.685514:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658215296 00000100:00100000:0.0:1787784838.685535:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658215296 00002000:00100000:0.0:1787784838.685607:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff989404dbf840 x1874627658215296/t0(0) o3->633bab11-ac8b-43e8-9332-f3266553ac37@192.168.203.43@tcp:719/0 lens 488/0 e 0 to 0 dl 1787784849 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784838.685622:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbf840 with x1874627658215296 ext(0->1048575) 00002000:00010000:0.0:1787784838.685629:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989404dbf840 00000100:00100000:0.0:1787784838.685680:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 112 00000100:00100000:0.0:1787784838.685693:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658215296:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1787784838.720661:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658215296:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 34953us (40588us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1787784838.720679:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 112 00002000:00100000:0.0:1787784838.720697:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff989404dbf840 x1874627658215296/t0(0) o3->633bab11-ac8b-43e8-9332-f3266553ac37@192.168.203.43@tcp:719/0 lens 488/440 e 0 to 0 dl 1787784849 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784838.720709:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbf840 with x1874627658215296 ext(0->1048575) 00002000:00010000:0.0:1787784838.720717:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989404dbf840 00000100:00100000:3.0:1787784838.756109:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784838.757108:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658215424 00000100:00100000:3.0:1787784838.757123:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658215424 00000100:00100000:3.0:1787784838.757255:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1286 00000100:00100000:3.0:1787784838.757267:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+9:30801:x1874627658215424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784838.757295:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784838.757320:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6e24 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1490 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.757363:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.757391:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.757405:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.758692:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc6e24 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 11 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.758725:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 10 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.758735:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:3.0:1787784838.758738:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784838.758743:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784838.758825:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+10:30801:x1874627658215424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1559us (2729us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784838.758834:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1286 00000100:00100000:3.0:1787784838.764229:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784838.764344:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658215552 00000100:00100000:0.0:1787784838.764367:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658215552 00000100:00100000:0.0:1787784838.764447:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1287 00000100:00100000:0.0:1787784838.764460:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+10:30801:x1874627658215552:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784838.764505:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784838.764878:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+10:30801:x1874627658215552:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 418us (674us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784838.764887:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1287 00000100:00100000:2.0:1787784838.771420:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784838.771469:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658215680 00000100:00100000:2.0:1787784838.771483:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658215680 00000100:00100000:2.0:1787784838.771699:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 357 00000100:00100000:2.0:1787784838.771711:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:2356:x1874627658215680:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784838.771767:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771790:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784838.771796:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784838.771804:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771821:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771864:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771875:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771885:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771893:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784838.771898:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771909:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771929:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.771937:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e32 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784838.772034:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:2356:x1874627658215680:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 325us (627us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784838.772043:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 357 00000100:00100000:1.0:1787784838.776438:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784838.776658:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658215936 00000100:00100000:0.0:1787784838.776672:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658215936 00000100:00100000:0.0:1787784838.776747:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1288 00000100:00100000:0.0:1787784838.776759:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:30801:x1874627658215936:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784838.776863:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784838.776884:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 10 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784838.776895:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784838.776939:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 10 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784838.776956:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 10 pid: 5812 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784838.776974:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668427776:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784838.777087:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989540180000 going to sleep for 11 seconds 00000100:00100000:1.0:1787784838.784936:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668427776:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784838.784977:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784838.785417:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784838.785604:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658216064 00000100:00100000:3.0:1787784838.785613:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658216064 00010000:00010000:3.0:1787784838.785673:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 11 pid: 5812 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784838.785847:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 382 00000100:00100000:3.0:1787784838.785861:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:633bab11-ac8b-43e8-9332-f3266553ac37+11:2357:x1874627658216064:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784838.785881:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784838.785890:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 11 pid: 5812 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.785905:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 11 pid: 5812 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.785932:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6e2b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1490 expref: 11 pid: 5812 timeout: 729 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.785953:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:3.0:1787784838.785960:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.785972:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.785981:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784838.786020:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784838.786182:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:633bab11-ac8b-43e8-9332-f3266553ac37+10:2357:x1874627658216064:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 323us (766us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784838.786193:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 382 00010000:00010000:2.0:1787784838.786486:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786497:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786751:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786760:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786767:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786774:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784838.786779:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786788:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786800:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:2.0:1787784838.786804:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784838.786809:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784838.786819:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e39 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784838.786996:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:30801:x1874627658215936:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 10236us (10572us total) trans 4294968147 rc 0/0 00000100:00100000:2.0:1787784838.787006:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1288 00000040:00080000:3.0:1787784838.787479:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784838.787486:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784838.788003:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:3.0:1787784838.788015:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784838.792989:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784838.793180:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658216320 00000100:00100000:3.0:1787784838.793192:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658216320 00000100:00100000:3.0:1787784838.793267:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 258 00000100:00100000:3.0:1787784838.793279:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+10:30801:x1874627658216320:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784838.793977:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+11:30801:x1874627658216320:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 696us (998us total) trans 4294968148 rc 0/0 00000100:00100000:3.0:1787784838.793987:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 258 00010000:00010000:0.0:1787784839.098672:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784839.190134:0:21570:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784839.190181:0:21570:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784839.190256:0:21570:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 38 count 5 00000040:00080000:3.0:1787784839.190263:0:21570:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784839.318992:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784839.322320:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658216448 00000100:00100000:3.0:1787784839.322342:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658216448 00000100:00100000:3.0:1787784839.322478:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 358 00000100:00100000:3.0:1787784839.322491:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658216448:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784839.322514:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784839.322535:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1497 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.322568:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784839.322574:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784839.322587:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.322608:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.322625:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784839.322715:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658216448:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 228us (3737us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784839.322723:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 358 00000100:00080000:1.0:1787784839.324251:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784839.335100:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.335414:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658216704 00000100:00100000:0.0:1787784839.335427:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658216704 00000100:00100000:0.0:1787784839.335521:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1289 00000100:00100000:0.0:1787784839.335535:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30802:x1874627658216704:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784839.335563:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784839.335589:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e47 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b149e expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.335734:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e4e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.335753:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e4e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.335769:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e4e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784839.335784:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:0.0:1787784839.336067:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc6e47 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b149e expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.336090:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e4e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b149e expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.336101:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:0.0:1787784839.336104:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784839.336108:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784839.336337:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30802:x1874627658216704:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 805us (1251us total) trans 4294968149 rc 0/0 00000100:00100000:0.0:1787784839.336346:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1289 00000100:00080000:3.0:1787784839.336987:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784839.344085:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784839.345967:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658216832 00000100:00100000:2.0:1787784839.345983:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658216832 00000100:00100000:2.0:1787784839.346066:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 359 00000100:00100000:2.0:1787784839.346078:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658216832:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:2.0:1787784839.346141:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.346190:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 5 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.346199:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.346222:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784839.346227:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784839.346233:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 5/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.346281:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.346300:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 17 pid: 6671 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784839.346317:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45cb40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668427904:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784839.346717:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 5 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.346730:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 5 pid: 10688 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784839.346742:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45da40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668428032:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784839.346771:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9c180 going to sleep for 11 seconds 00000100:00100000:2.0:1787784839.371631:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784839.371694:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.371766:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658216960 00000100:00100000:0.0:1787784839.371778:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658216960 00010000:00010000:0.0:1787784839.371842:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 18 pid: 6671 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784839.371893:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658217088 00000100:00100000:0.0:1787784839.371896:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658217088 00010000:00010000:0.0:1787784839.371910:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 6 pid: 10688 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784839.371931:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 383 00000100:00100000:0.0:1787784839.371942:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2356:x1874627658216960:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784839.371963:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784839.371991:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 18 pid: 6671 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.372007:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 18 pid: 6671 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.372040:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784839.372104:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2356:x1874627658216960:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 164us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784839.372112:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 383 00000100:00100000:0.0:1787784839.372139:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 384 00000100:00100000:0.0:1787784839.372179:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895089b2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658217088:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784839.372190:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784839.372200:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 6 pid: 10688 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.372225:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 6 pid: 10688 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.372240:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784839.372245:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784839.372252:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.372263:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.372286:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784839.372307:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895089b2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658217088:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 161us (613us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784839.372313:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 384 00010000:00010000:3.0:1787784839.372767:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005044093d/0xb0ce63137acc6e40 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1497 expref: 17 pid: 6671 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784839.372785:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45cb40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668427904:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784839.372801:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc6e1d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1489 expref: 5 pid: 10688 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784839.372811:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45da40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668428032:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784839.372849:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372861:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372873:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372904:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372912:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372920:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372927:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784839.372932:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372941:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372953:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.372960:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488b76e5/0xb0ce63137acc6e5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784839.373039:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658216832:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 26964us (28968us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784839.373047:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 359 00000100:00100000:0.0:1787784839.384992:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784839.385280:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658217344 00000100:00100000:3.0:1787784839.385294:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658217344 00000100:00100000:3.0:1787784839.385360:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 259 00000100:00100000:3.0:1787784839.385372:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30802:x1874627658217344:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:3.0:1787784839.385457:0:5815:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x27:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e4e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b149e expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784839.385749:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968150 00000001:00080000:3.0:1787784839.386580:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968151 00010000:00010000:3.0:1787784839.386655:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386678:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386689:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 105 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386698:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 10 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386708:0:5815:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 6/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386758:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 10 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386777:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 10 pid: 5812 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784839.386795:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668428160:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784839.386917:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 105 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386929:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 105 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784839.386938:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668428288:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784839.386968:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 47 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.386978:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 47 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784839.386987:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668428416:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784839.387008:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784839.392702:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668428160:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784839.392721:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668428288:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784839.392733:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668428416:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784839.392772:0:5815:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784839.405242:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.405285:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.405319:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784839.405404:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658217472 00000100:00100000:1.0:1787784839.405418:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658217472 00010000:00010000:1.0:1787784839.405477:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 11 pid: 5812 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784839.405518:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658217600 00000100:00100000:1.0:1787784839.405521:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658217600 00010000:00010000:1.0:1787784839.405535:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 106 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784839.405546:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658217728 00000100:00100000:1.0:1787784839.405549:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658217728 00010000:00010000:1.0:1787784839.405580:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 48 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784839.405606:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 385 00000100:00100000:3.0:1787784839.405616:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:1.0:1787784839.405618:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:633bab11-ac8b-43e8-9332-f3266553ac37+11:2355:x1874627658217472:12345-192.168.203.43@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787784839.405630:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2354:x1874627658217600:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784839.405640:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784839.405642:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784839.405652:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 11 pid: 5812 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.405653:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 106 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.405680:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 11 pid: 5812 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784839.405719:0:5795:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[0] started 4 min 3 max 56 00000100:00100000:0.0:1787784839.405729:0:5795:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn00_004' 00010000:00010000:3.0:1787784839.405772:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 106 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.405801:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6dd7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b145f expref: 106 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.405823:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:3.0:1787784839.405829:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.405839:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784839.405848:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784839.405906:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 387 00000100:00100000:3.0:1787784839.405919:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:2354:x1874627658217600:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 291us (630us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784839.405919:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2354:x1874627658217728:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784839.405933:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787784839.405934:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 386 00010000:00010000:0.0:1787784839.405941:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 48 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.405955:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 48 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.405972:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6d67 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b143c expref: 48 pid: 5811 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787784839.405973:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784839.405987:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:0.0:1787784839.405992:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.406000:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.406005:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784839.406025:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2354:x1874627658217728:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 111us (705us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784839.406031:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 387 00010000:00010000:1.0:1787784839.406203:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc6e08 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1482 expref: 11 pid: 5812 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.406224:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:1.0:1787784839.406228:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.406236:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.406244:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.406280:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784839.406300:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:633bab11-ac8b-43e8-9332-f3266553ac37+10:2355:x1874627658217472:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 686us (1076us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784839.406308:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 385 00010000:00010000:0.0:1787784839.406911:0:5815:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.406921:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784839.407092:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968152 00000001:00080000:0.0:1787784839.407395:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968153 00000001:00080000:0.0:1787784839.407455:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968154 00000100:00100000:0.0:1787784839.407562:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668428544:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1787784839.407633:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784839.407671:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784839.408241:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668428544 00000100:00100000:1.0:1787784839.408254:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668428544 00000100:00100000:1.0:1787784839.408310:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 61 00000100:00100000:1.0:1787784839.408322:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668428544:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1787784839.408426:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668428544:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 106us (793us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784839.408434:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 61 00000100:00100000:3.0:1787784839.408912:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668428544:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:3.0:1787784839.409767:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968155 00000100:00100000:3.0:1787784839.410646:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668428672:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1787784839.410712:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784839.410753:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1480 going to sleep for 11 seconds 00000100:00100000:0.0:1787784839.410853:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668428672 00000100:00100000:0.0:1787784839.410862:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668428672 00000100:00100000:0.0:1787784839.410925:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 62 00000100:00100000:0.0:1787784839.410938:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668428672:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1787784839.411326:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668428672:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 390us (612us total) trans 4294967366 rc 0/0 00000100:00100000:0.0:1787784839.411337:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 62 00000100:00100000:0.0:1787784839.411404:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895402883c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668428672:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:0.0:1787784839.411432:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.411445:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.411453:0:5815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.411460:0:5815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784839.411465:0:5815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.411475:0:5815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.411494:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:0.0:1787784839.411498:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1787784839.411505:0:5815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.411512:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc6e63 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784839.411781:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x4c:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1787784839.411961:0:5815:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].11, 64 off8920 00000004:00080000:0.0:1787784839.411974:0:5815:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 630498826227ns, next at 631498823192ns, rc = 0 00000001:00080000:0.0:1787784839.411983:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968156 00010000:00010000:0.0:1787784839.413257:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520131300) 00010000:00010000:0.0:1787784839.413268:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784839.413280:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e4e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b149e expref: 109 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784839.416191:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30802:x1874627658217344:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 30808us (31201us total) trans 4294968157 rc 0/0 00000100:00100000:0.0:1787784839.416221:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 259 00000020:00080000:0.0:1787784839.416231:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:0.0:1787784839.474129:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784839.474635:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658217984 00000100:00100000:1.0:1787784839.474649:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658217984 00000100:00100000:1.0:1787784839.474726:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1290 00000100:00100000:1.0:1787784839.474739:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:30805:x1874627658217984:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784839.474769:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784839.474796:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6e6a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14a5 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.474891:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.474909:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.474922:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.474940:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.474947:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.474954:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.475906:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115800/0xb0ce63137acc6e7f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.475923:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115800/0xb0ce63137acc6e7f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.475932:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115800/0xb0ce63137acc6e7f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784839.475944:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:1.0:1787784839.476053:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.476061:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.476070:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.476077:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784839.476082:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.476090:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.476104:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:1.0:1787784839.476107:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784839.476112:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.476125:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c600/0xb0ce63137acc6e71 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.476139:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477411:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477436:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477443:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404e1c000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784839.477448:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477458:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477472:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff98950a38c300) 00010000:00010000:1.0:1787784839.477474:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784839.477478:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477483:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1c000/0xb0ce63137acc6e78 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477549:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1cc00/0xb0ce63137acc6e6a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14a5 expref: 111 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477565:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115800/0xb0ce63137acc6e7f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14a5 expref: 110 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784839.477570:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784839.477572:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784839.477574:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98952a115800, rc 0) 00000100:00100000:1.0:1787784839.479565:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:30805:x1874627658217984:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4812us (5431us total) trans 4294968158 rc 0/0 00000100:00100000:1.0:1787784839.479582:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1290 00000100:00100000:0.0:1787784839.495764:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784839.495920:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658218112 00000100:00100000:2.0:1787784839.495933:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658218112 00000100:00100000:2.0:1787784839.496007:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1291 00000100:00100000:2.0:1787784839.496019:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:30805:x1874627658218112:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:2.0:1787784839.496109:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496131:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496168:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496218:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496226:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496233:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496240:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784839.496246:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496255:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496268:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784839.496272:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784839.496278:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.496284:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6e8d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784839.496374:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:30805:x1874627658218112:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 357us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784839.496384:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1291 00000100:00100000:2.0:1787784839.500265:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.500348:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658218368 00000100:00100000:0.0:1787784839.500362:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658218368 00000100:00100000:0.0:1787784839.500437:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 260 00000100:00100000:0.0:1787784839.500450:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:30805:x1874627658218368:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:0.0:1787784839.500758:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:30805:x1874627658218368:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 309us (502us total) trans 4294968159 rc 0/0 00000100:00100000:0.0:1787784839.500767:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 260 00000100:00100000:2.0:1787784839.680928:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.681138:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658218496 00000100:00100000:0.0:1787784839.681318:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658218496 00000100:00100000:0.0:1787784839.681431:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1292 00000100:00100000:0.0:1787784839.681444:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+10:30780:x1874627658218496:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1787784839.681948:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+10:30780:x1874627658218496:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 506us (1027us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784839.681957:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1292 00000100:00100000:2.0:1787784839.685329:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784839.685620:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658218752 00000100:00100000:1.0:1787784839.685638:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658218752 00000100:00100000:1.0:1787784839.685924:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 261 00000100:00100000:1.0:1787784839.685941:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:30780:x1874627658218752:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784839.686630:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+11:30780:x1874627658218752:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 693us (1305us total) trans 4294968160 rc 0/0 00000100:00100000:1.0:1787784839.686642:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 261 00000001:00100000:1.0:1787784839.711101:0:7630:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 633bab11-ac8b-43e8-9332-f3266553ac37 00000100:00100000:1.0:1787784839.909034:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.909329:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658219008 00000100:00100000:0.0:1787784839.909346:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658219008 00000100:00100000:0.0:1787784839.909427:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1293 00000100:00100000:0.0:1787784839.909438:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:30826:x1874627658219008:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784839.909471:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784839.909498:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6e94 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14ac expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.909643:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e9b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.909660:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e9b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.909673:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e9b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784839.909686:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:0.0:1787784839.909855:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6e94 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ac expref: 114 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.909889:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e9b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ac expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.910520:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784839.910528:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784839.910532:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784839.910702:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:30826:x1874627658219008:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1263us (1679us total) trans 4294968161 rc 0/0 00000100:00100000:0.0:1787784839.910712:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1293 00000100:00100000:3.0:1787784839.922833:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784839.923699:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658219264 00000100:00100000:2.0:1787784839.923713:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658219264 00000100:00100000:2.0:1787784839.923793:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 56 00000100:00100000:2.0:1787784839.923805:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:30826:x1874627658219264:12345-192.168.203.43@tcp:103:cat.0 00010000:00010000:2.0:1787784839.923828:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1787784839.923864:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6e9b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ac expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.923952:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115800/0xb0ce63137acc6e7f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14a5 expref: 112 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.923966:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784839.924071:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784839.924520:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:30826:x1874627658219264:12345-192.168.203.43@tcp:103:cat.0 Request processed in 716us (1701us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784839.924531:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 56 00000100:00100000:2.0:1787784839.928532:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.928658:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658219392 00000100:00100000:0.0:1787784839.928669:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658219392 00000100:00100000:0.0:1787784839.928740:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1294 00000100:00100000:0.0:1787784839.928751:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:30826:x1874627658219392:12345-192.168.203.43@tcp:58:cat.0 00010000:00010000:0.0:1787784839.928896:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.928917:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784839.928930:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784839.929187:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].38, 136 off13360 00000100:00100000:0.0:1787784839.929637:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:30826:x1874627658219392:12345-192.168.203.43@tcp:58:cat.0 Request processed in 886us (1114us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784839.929647:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1294 00000100:00100000:1.0:1787784839.933913:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784839.935118:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658219520 00000100:00100000:0.0:1787784839.935134:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658219520 00000100:00100000:2.0:1787784839.935422:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1295 00000100:00100000:2.0:1787784839.935437:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:30826:x1874627658219520:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784839.935465:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784839.935492:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14b3 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.935572:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.935585:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14b3 expref: 112 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.935610:0:5811:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.935624:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14b3 expref: 112 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.935641:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:2.0:1787784839.935645:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784839.935649:0:5811:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14b3 expref: 112 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784839.935658:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2600, rc 0) 00000100:00100000:2.0:1787784839.935743:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:30826:x1874627658219520:12345-192.168.203.43@tcp:101:cat.0 Request processed in 309us (1852us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784839.935752:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1295 00010000:00010000:0.0:1787784840.114956:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784840.689628:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668428800:0@lo:400:kworker.0 00000100:00100000:0.0:1787784840.689726:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668429056:0@lo:400:kworker.0 00000100:00100000:1.0:1787784840.689735:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784840.689939:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668428800 00000100:00100000:3.0:1787784840.689949:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668428800 00000100:00100000:1.0:1787784840.689959:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668429312:0@lo:400:kworker.0 00000100:00100000:1.0:1787784840.689995:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784840.690013:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 594 00000100:00100000:1.0:1787784840.690026:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1787784840.690027:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668428800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784840.690038:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668428928:0@lo:400:kworker.0 00000100:00100000:1.0:1787784840.690066:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784840.690088:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668429440:0@lo:400:kworker.0 00000100:00100000:3.0:1787784840.690104:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668428800:12345-0@lo:400:kworker.0 Request processed in 79us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784840.690112:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 594 00000100:00100000:1.0:1787784840.690121:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784840.690192:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668428800:0@lo:400:kworker.0 00000100:00100000:1.0:1787784840.690240:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429440 00000100:00100000:1.0:1787784840.690244:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429440 00000100:00100000:3.0:1787784840.690302:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539100b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668429184:0@lo:400:kworker.0 00000100:00100000:3.0:1787784840.690335:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784840.690390:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429312 00000100:00100000:2.0:1787784840.690395:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429312 00000100:00100000:2.0:1787784840.690438:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668428928 00000100:00100000:2.0:1787784840.690441:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668428928 00000100:00100000:2.0:1787784840.690461:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1787784840.690468:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668429312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784840.690517:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668429312:12345-0@lo:400:kworker.0 Request processed in 49us (521us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784840.690525:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:3.0:1787784840.690534:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429184 00000100:00100000:3.0:1787784840.690539:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429184 00000100:00100000:2.0:1787784840.690543:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:2.0:1787784840.690551:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668428928:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784840.690585:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1787784840.690586:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668428928:12345-0@lo:400:kworker.0 Request processed in 33us (516us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784840.690598:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1787784840.690600:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668429184:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787784840.690620:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784840.690626:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784840.690630:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784840.690642:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1787784840.690652:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668429184:12345-0@lo:400:kworker.0 Request processed in 57us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784840.690654:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668429440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784840.690665:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1787784840.690691:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539100b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668429184:0@lo:400:kworker.0 00000100:00100000:1.0:1787784840.690703:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668429440:12345-0@lo:400:kworker.0 Request processed in 52us (580us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784840.690710:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:2.0:1787784840.690747:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784840.690752:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668428928:0@lo:400:kworker.0 00000100:00100000:2.0:1787784840.690757:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668429568:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784840.690767:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668429440:0@lo:400:kworker.0 00000100:00100000:1.0:1787784840.690779:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668429312:0@lo:400:kworker.0 00000100:00100000:2.0:1787784840.690794:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784840.691388:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429568 00000100:00100000:3.0:1787784840.691395:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429568 00000100:00100000:3.0:1787784840.691442:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1787784840.691451:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668429568:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784840.691485:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784840.691539:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668429568:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (747us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784840.691546:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1787784840.691691:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668429568:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784840.694171:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784840.694275:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429056 00000100:00100000:2.0:1787784840.694286:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429056 00000100:00100000:2.0:1787784840.694384:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00080000:3.0:1787784840.694389:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784840.694398:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668429056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784840.695786:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668429056:12345-0@lo:400:kworker.0 Request processed in 1370us (1617us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784840.695800:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1787784840.704885:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bfc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668429056:0@lo:400:kworker.0 00000040:00080000:3.0:1787784840.818644:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784840.818656:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784840.818707:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 6 00000040:00080000:3.0:1787784840.818718:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784841.137997:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784841.992906:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784841.992962:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784841.993021:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784841.993055:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784841.993083:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784841.993133:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784841.993231:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658219904 00000100:00100000:2.0:1787784841.993244:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658219904 00000100:00100000:1.0:1787784841.993384:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220160 00000100:00100000:1.0:1787784841.993389:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220160 00000100:00100000:1.0:1787784841.993470:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658219776 00000100:00100000:1.0:1787784841.993474:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658219776 00000100:00100000:1.0:1787784841.993502:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:1.0:1787784841.993514:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658220160:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784841.993626:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2356:x1874627658220160:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 114us (604us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784841.993636:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:1.0:1787784841.993657:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 446 00000100:00100000:1.0:1787784841.993664:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:2356:x1874627658219776:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784841.993692:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:2356:x1874627658219776:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (560us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784841.993698:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 446 00000100:00080000:1.0:1787784841.993716:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784841.993863:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220416 00000100:00100000:1.0:1787784841.993868:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220416 00000100:00100000:1.0:1787784841.993900:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220672 00000100:00100000:1.0:1787784841.993903:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220672 00000100:00100000:1.0:1787784841.993922:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00100000:1.0:1787784841.993929:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658220416:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784841.993964:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658220416:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 35us (909us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784841.993971:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00100000:1.0:1787784841.993983:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:1.0:1787784841.993989:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2356:x1874627658220672:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784841.994011:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2356:x1874627658220672:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (929us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784841.994017:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00080000:1.0:1787784841.994037:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784841.994040:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784841.995131:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:2.0:1787784841.995143:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658219904:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784841.995231:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658219904:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 89us (2267us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784841.995238:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:3.0:1787784841.996011:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658219648 00000100:00100000:3.0:1787784841.996023:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658219648 00000100:00100000:3.0:1787784841.996077:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 595 00000100:00100000:3.0:1787784841.996087:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:2356:x1874627658219648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784841.996143:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:2356:x1874627658219648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 58us (3252us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784841.996203:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 595 00000100:00100000:0.0:1787784841.996280:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784841.996752:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220032 00000100:00100000:3.0:1787784841.996757:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220032 00000100:00100000:3.0:1787784841.996819:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 407 00000100:00100000:3.0:1787784841.996828:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658220032:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:1.0:1787784841.996858:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784841.996864:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784841.997082:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:2356:x1874627658220032:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 253us (802us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784841.997091:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 407 00000100:00100000:0.0:1787784842.012637:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784842.012800:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220288 00000100:00100000:2.0:1787784842.012814:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220288 00000100:00100000:2.0:1787784842.012915:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 408 00000100:00100000:2.0:1787784842.012927:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658220288:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784842.013030:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658220288:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 106us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784842.013038:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 408 00000100:00080000:1.0:1787784842.013897:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784842.027452:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784842.027512:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784842.027674:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220544 00000100:00100000:1.0:1787784842.027686:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220544 00000100:00100000:1.0:1787784842.027774:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00100000:1.0:1787784842.027787:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+11:2356:x1874627658220544:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784842.027895:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+11:2356:x1874627658220544:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 112us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784842.027903:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00080000:0.0:1787784842.028090:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784842.028544:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220800 00000100:00100000:3.0:1787784842.028551:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220800 00000100:00100000:3.0:1787784842.028611:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 409 00000100:00100000:3.0:1787784842.028620:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527debc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2356:x1874627658220800:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784842.028668:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527debc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2356:x1874627658220800:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 50us (1155us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784842.028675:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 409 00000100:00080000:0.0:1787784842.029291:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787784842.163728:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784842.773536:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968161 is committed 00000004:00080000:1.0:1787784842.773556:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 13 committed 00000001:00080000:1.0:1787784842.773595:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968160 is committed 00000020:00080000:1.0:1787784842.773620:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9895401a56c0 refcount 2 committed 0 result 0 batchid 29 00000020:00080000:1.0:1787784842.773626:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989540610b00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9895401a5e40 00000020:00080000:1.0:1787784842.773629:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989540610f00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9894035ec400 00000001:00080000:1.0:1787784842.773657:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968145 is committed 00000020:00080000:3.0:1787784842.773712:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 29 committed batchid 28 00000020:00080000:3.0:1787784842.773881:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 28 00000100:00100000:3.0:1787784842.774144:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668429696:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784842.774286:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784842.774399:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429696 00000100:00100000:3.0:1787784842.774408:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429696 00000100:00100000:3.0:1787784842.774482:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1787784842.774492:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668429696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787784842.774526:0:6669:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:25 count 1 00002000:00010000:3.0:1787784842.774569:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 1048576, mtime: 0x6a8f6e84, atime: 0x0, ctime: 0x6a8f6e84, blocks: 0x803 00010000:00010000:3.0:1787784842.774580:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784842.774603:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=633s v=(0 0) 00010000:00010000:3.0:1787784842.774608:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784842.774616:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784842.774633:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784842.774652:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784842.774661:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784842.774688:0:6669:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787784842.774789:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784842.774800:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784842.774812:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784842.774817:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784842.774830:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784842.774855:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784842.774865:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059ecaec7/0xb0ce63137acc6eb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784842.774893:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784842.774896:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784842.774899:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787784842.775047:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784842.775290:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668429696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 797us (1004us total) trans 4294967367 rc 0/0 00000100:00100000:3.0:1787784842.775299:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000004:00080000:0.0:1787784842.775496:0:3277:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401fa7840/2, rc 0, transno 4294967367 00000100:00100000:0.0:1787784842.775503:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668429696:0@lo:6:osp-syn-0-0.0 00000001:00080000:0.0:1787784842.780569:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967366 is committed 00000040:00080000:3.0:1787784842.865672:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784842.865684:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784842.866271:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 3 00000040:00080000:3.0:1787784842.866283:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784843.185622:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784843.761527:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668429824:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784843.761604:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784843.761800:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429824 00000100:00100000:3.0:1787784843.761808:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429824 00000100:00100000:3.0:1787784843.761858:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1787784843.761865:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668429824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784843.761895:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1787784843.761961:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668429824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784843.761968:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1787784843.763068:0:3278:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967366 gen 2 00000004:00080000:2.0:1787784843.763076:0:3278:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967366 committed transno 4294967366 00000100:00100000:2.0:1787784843.763102:0:3278:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff989401fa7840 x1874627668429696/t4294967367(4294967367) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1787784858 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787784843.763121:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668429824:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787784844.209502:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784844.923935:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784844.923982:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658220928 00000100:00100000:1.0:1787784844.923990:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658220928 00000100:00100000:1.0:1787784844.924115:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1296 00000100:00100000:1.0:1787784844.924128:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30829:x1874627658220928:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:1.0:1787784844.924164:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784844.924195:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6ebe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14ba expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787784844.924229:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784844.924262:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924277:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924290:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924305:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924312:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924319:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924659:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924670:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x53/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924678:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x53/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784844.924714:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:1.0:1787784844.924719:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784844.924741:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x53/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924752:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924765:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924772:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924778:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924784:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a114c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784844.924788:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924796:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924808:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:1.0:1787784844.924811:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784844.924815:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924822:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6ec5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924834:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924840:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924845:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924851:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784844.924854:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924860:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924867:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff989519d44600) 00010000:00010000:1.0:1787784844.924869:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784844.924872:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924878:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6ecc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924909:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6ebe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924924:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784844.924930:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:1.0:1787784844.924932:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784844.924935:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98952a114000, rc 0) 00000100:00100000:1.0:1787784844.925001:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30829:x1874627658220928:12345-192.168.203.43@tcp:101:stat.0 Request processed in 876us (1074us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784844.925009:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1296 00010000:00010000:2.0:1787784845.233337:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784845.273529:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784845.273814:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658221056 00000100:00100000:0.0:1787784845.273823:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658221056 00000100:00100000:0.0:1787784845.273910:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1297 00000100:00100000:0.0:1787784845.273921:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+7:30853:x1874627658221056:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784845.273941:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784845.273962:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc6eda lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14c1 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784845.274028:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6ee1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784845.274041:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784845.274053:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784845.274450:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00200000:0.0:1787784845.275977:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784845.275987:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 31 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784845.276133:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784845.276204:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc6eda lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14c1 expref: 9 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784845.276238:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6ee1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14c1 expref: 8 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784845.276248:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784845.276251:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784845.276255:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784845.276334:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+8:30853:x1874627658221056:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2415us (2814us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784845.276342:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1297 00000100:00100000:0.0:1787784845.295991:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784845.296107:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658221312 00000100:00100000:2.0:1787784845.296116:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658221312 00000100:00100000:2.0:1787784845.296211:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1298 00000100:00100000:2.0:1787784845.296220:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+8:30854:x1874627658221312:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784845.296330:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784845.296351:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6ee8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14c8 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.296590:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.296605:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.296621:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.296635:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.296643:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.296649:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.297391:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6efd lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.297548:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6efd lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.297556:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6efd lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784845.297566:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 6656 00010000:00010000:2.0:1787784845.298473:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298483:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298488:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298492:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784845.298495:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298501:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298510:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:2.0:1787784845.298512:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784845.298515:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298519:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6eef lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298528:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298532:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298535:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298538:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784845.298539:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298543:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298546:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff989520131700) 00010000:00010000:2.0:1787784845.298547:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784845.298550:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298553:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6ef6 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298974:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6ee8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14c8 expref: 11 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298985:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6efd lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14c8 expref: 10 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.298989:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784845.298991:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784845.298993:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:2.0:1787784845.299103:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+10:30854:x1874627658221312:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2884us (3121us total) trans 4294968162 rc 0/0 00000100:00100000:2.0:1787784845.299110:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1298 00000100:00100000:2.0:1787784845.305137:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784845.305235:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658221440 00000100:00100000:1.0:1787784845.305245:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658221440 00000100:00100000:1.0:1787784845.305299:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1299 00000100:00100000:1.0:1787784845.305309:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+10:30854:x1874627658221440:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1787784845.305428:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+10:30854:x1874627658221440:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 120us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784845.305436:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1299 00000100:00100000:3.0:1787784845.309585:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784845.309688:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658221568 00000100:00100000:0.0:1787784845.309698:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658221568 00000100:00100000:0.0:1787784845.309765:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 57 00000100:00100000:0.0:1787784845.309776:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:2357:x1874627658221568:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784845.309793:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784845.309817:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc6efd lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14c8 expref: 10 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784845.309835:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784845.309840:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784845.310061:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:633bab11-ac8b-43e8-9332-f3266553ac37+9:2357:x1874627658221568:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 286us (483us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784845.310069:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 57 00000100:00100000:2.0:1787784845.809358:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784845.809382:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401835680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668429952:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784845.809449:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784845.809520:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668430336:0@lo:400:kworker.0 00000100:00100000:1.0:1787784845.809543:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668430080:0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.809552:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784845.809579:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784845.809588:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668430464:0@lo:400:kworker.0 00000100:00100000:1.0:1787784845.809589:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784845.809619:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668430592:0@lo:400:kworker.0 00000100:00100000:1.0:1787784845.809642:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784845.809663:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784845.809669:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668430208:0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.809688:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784845.809700:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784845.809728:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668429952 00000100:00100000:2.0:1787784845.809736:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668429952 00000100:00100000:1.0:1787784845.809739:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668430080 00000100:00100000:1.0:1787784845.809744:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668430080 00000100:00100000:1.0:1787784845.809784:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 596 00000100:00100000:2.0:1787784845.809790:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:1.0:1787784845.809795:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668430080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.809802:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668429952:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784845.809831:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784845.809848:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668430080:12345-0@lo:400:kworker.0 Request processed in 55us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784845.809856:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 596 00000100:00100000:2.0:1787784845.809879:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668429952:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784845.809887:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:1.0:1787784845.809892:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668430336 00000100:00100000:1.0:1787784845.809897:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668430336 00000100:00100000:2.0:1787784845.809912:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668430464 00000100:00100000:2.0:1787784845.809917:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668430464 00000100:00100000:1.0:1787784845.809927:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668430592 00000100:00100000:1.0:1787784845.809932:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668430592 00000100:00100000:2.0:1787784845.809952:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1787784845.809959:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668430208 00000100:00100000:2.0:1787784845.809963:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668430464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784845.809964:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668430208 00000100:00100000:1.0:1787784845.809984:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1787784845.809991:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668430336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.810003:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668430464:12345-0@lo:400:kworker.0 Request processed in 42us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784845.810011:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1787784845.810034:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668430336:12345-0@lo:400:kworker.0 Request processed in 42us (481us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784845.810041:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1787784845.810049:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668430592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.810079:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668430592:12345-0@lo:400:kworker.0 Request processed in 31us (435us total) trans 0 rc 0/0 00000100:00080000:3.0:1787784845.810211:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784845.810217:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784845.810219:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784845.811045:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1787784845.811067:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1787784845.811077:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668430208:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784845.811119:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668430208:12345-0@lo:400:kworker.0 Request processed in 45us (1419us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784845.811126:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1787784845.811201:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668430208:0@lo:400:kworker.0 00000100:00100000:1.0:1787784845.811216:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668430080:0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.811217:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1787784845.811233:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668430592:0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.811267:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668430464:0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.811281:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668430336:0@lo:400:kworker.0 00000100:00100000:2.0:1787784845.811302:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401835680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668429952:0@lo:13:osp-pre-1-0.0 00000040:00080000:2.0:1787784845.937524:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784845.937533:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784845.937579:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 3 00000040:00080000:2.0:1787784845.937587:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784845.938120:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668430720:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784845.938303:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401806c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784845.949381:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784845.950333:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658221696 00000100:00100000:1.0:1787784845.950344:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658221696 00000100:00100000:1.0:1787784845.950413:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1300 00000100:00100000:1.0:1787784845.950423:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+10:30855:x1874627658221696:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784845.950457:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784845.950566:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+10:30855:x1874627658221696:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 144us (1205us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784845.950573:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1300 00000100:00100000:2.0:1787784845.951911:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668430720:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784845.958621:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784845.958977:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658221824 00000100:00100000:1.0:1787784845.958988:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658221824 00000100:00100000:1.0:1787784845.959046:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1301 00000100:00100000:1.0:1787784845.959057:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+9:30855:x1874627658221824:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1787784845.959132:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959189:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959204:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959830:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959842:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959849:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959873:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959881:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959887:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784845.959923:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:1.0:1787784845.959927:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784845.959947:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959957:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959964:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959971:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959977:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784845.959981:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959988:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.959998:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784845.960001:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784845.960005:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960010:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6f0b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960022:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960027:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960033:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960038:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784845.960041:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960047:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960053:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x4e3552f8 (ffff989519d44600) 00010000:00010000:1.0:1787784845.960055:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784845.960058:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960064:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115600/0xb0ce63137acc6f12 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960078:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960083:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960088:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960093:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a114c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784845.960096:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960102:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960107:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:1.0:1787784845.960110:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784845.960113:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784845.960118:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114c00/0xb0ce63137acc6f19 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784845.960402:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+9:30855:x1874627658221824:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 1344us (1787us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784845.960410:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1301 00000100:00100000:0.0:1787784845.975222:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784845.975716:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658222080 00000100:00100000:3.0:1787784845.975727:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658222080 00000100:00100000:3.0:1787784845.975801:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1302 00000100:00100000:3.0:1787784845.975812:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+9:30855:x1874627658222080:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784845.975833:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784845.975854:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6f20 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14cf expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784845.978266:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784845.978290:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784845.978302:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784845.978317:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:2.0:1787784845.979803:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.979818:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.979826:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.979833:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784845.979837:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.979844:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.979858:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:2.0:1787784845.979862:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784845.979866:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.979873:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc6f27 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.979930:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6f20 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14cf expref: 11 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.980858:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784845.980862:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784845.980869:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc6f20 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14cf expref: 11 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.980883:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:2.0:1787784845.981220:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+10:30855:x1874627658222080:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5408us (6006us total) trans 4294968163 rc 301/301 00000100:00100000:2.0:1787784845.981230:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1302 00000100:00100000:3.0:1787784845.989559:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784845.989864:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658222208 00000100:00100000:2.0:1787784845.989872:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658222208 00000100:00100000:2.0:1787784845.989969:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1303 00000100:00100000:2.0:1787784845.989979:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:30855:x1874627658222208:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784845.990016:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784845.990214:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:30855:x1874627658222208:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 235us (661us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784845.990222:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1303 00000100:00100000:0.0:1787784845.997530:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784845.997968:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658222336 00000100:00100000:2.0:1787784845.997978:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658222336 00000100:00100000:2.0:1787784845.998040:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1304 00000100:00100000:2.0:1787784845.998050:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+10:30855:x1874627658222336:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784845.998068:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784845.998090:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f35 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14d6 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.998516:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.998534:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.998546:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.998562:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.998568:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.998575:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999213:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6f4a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999226:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6f4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999233:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6f4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784845.999310:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 7680 00000004:00200000:2.0:1787784845.999315:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784845.999489:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6f4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999505:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999511:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999517:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999523:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784845.999527:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999534:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999544:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784845.999546:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784845.999550:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999555:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f3c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999566:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999572:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999577:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999582:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784845.999705:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999714:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999724:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff989520131700) 00010000:00010000:2.0:1787784845.999726:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784845.999729:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999735:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f43 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999771:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f35 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14d6 expref: 12 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999784:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6f4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14d6 expref: 11 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784845.999791:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:2.0:1787784845.999793:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784845.999795:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166e800, rc 0) 00000100:00100000:2.0:1787784845.999865:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658222336:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1816us (2343us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784845.999873:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1304 00000100:00100000:0.0:1787784846.004819:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.005062:0:5821:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658222464 00000100:00100000:2.0:1787784846.005072:0:5821:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658222464 00000100:00100000:2.0:1787784846.005126:0:5821:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 2 00000100:00100000:2.0:1787784846.005135:0:5821:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqm_0001:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658222464:12345-192.168.203.43@tcp:700:lhsmtool_posix.0 00000100:00100000:2.0:1787784846.006429:0:5821:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqm_0001:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658222464:12345-192.168.203.43@tcp:700:lhsmtool_posix.0 Request processed in 1288us (1612us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.006442:0:5821:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 2 00000100:00100000:3.0:1787784846.011870:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.011927:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658222720 00000100:00100000:2.0:1787784846.011937:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658222720 00000100:00100000:2.0:1787784846.011991:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1305 00000100:00100000:2.0:1787784846.012001:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658222720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784846.012022:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.012041:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f51 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14dd expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.013058:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6f58 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.013074:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6f58 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.013084:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6f58 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784846.013095:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1:0x0]: returning size 0 00010000:00010000:2.0:1787784846.013140:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f51 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14dd expref: 14 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.013192:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6f58 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14dd expref: 13 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.013207:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784846.013210:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784846.013214:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950166f800, rc 0) 00000100:00100000:2.0:1787784846.013271:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+13:30855:x1874627658222720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1272us (1409us total) trans 4294968164 rc 0/0 00000100:00100000:2.0:1787784846.013279:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1305 00000100:00100000:3.0:1787784846.019689:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.019749:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658222976 00000100:00100000:2.0:1787784846.019760:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658222976 00000100:00100000:2.0:1787784846.019825:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1306 00000100:00100000:2.0:1787784846.019836:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+13:30855:x1874627658222976:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784846.019892:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784846.019917:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6f58 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14dd expref: 13 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.019936:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784846.019945:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784846.019966:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.019980:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.019989:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.020018:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.020033:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 99 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.020048:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668430848:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784846.020129:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401800c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784846.022482:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668430848:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784846.022511:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.022542:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.022599:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658223104 00000100:00100000:1.0:1787784846.022606:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658223104 00010000:00010000:1.0:1787784846.022653:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 100 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787784846.022656:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784846.022686:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 388 00000100:00100000:1.0:1787784846.022695:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:2357:x1874627658223104:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784846.022707:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784846.022713:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 100 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.022726:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 100 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.022746:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404e1da00/0xb0ce63137acc6df3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1474 expref: 100 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.022761:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:1.0:1787784846.022766:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.022776:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.022785:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.022813:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784846.022841:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.022851:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.022866:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:2357:x1874627658223104:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 170us (324us total) trans 0 rc 0/0 00010000:00010000:3.0:1787784846.022878:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.022883:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 388 00010000:00010000:3.0:1787784846.022889:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.022896:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023220:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023229:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023235:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023242:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950166ec00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784846.023246:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023254:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023262:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:3.0:1787784846.023264:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784846.023268:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023274:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166ec00/0xb0ce63137acc6f66 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023285:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023297:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023303:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023306:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784846.023352:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023359:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023367:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xe4a4453b (ffff9895198b5200) 00010000:00010000:3.0:1787784846.023382:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784846.023386:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.023393:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f6d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xe4a4453b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.023454:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+12:30855:x1874627658222976:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 3619us (3774us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784846.023460:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1306 00000100:00100000:3.0:1787784846.025261:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.025292:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658223232 00000100:00100000:2.0:1787784846.025301:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658223232 00000100:00100000:2.0:1787784846.025348:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1307 00000100:00100000:2.0:1787784846.025358:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+12:30855:x1874627658223232:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784846.025375:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.025394:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6f74 lrc: 2/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14e4 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.025429:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.025441:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.025452:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.025540:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f800/0xb0ce63137acc6f74 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 14 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.025556:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 13 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.025565:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784846.025567:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784846.025570:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:2.0:1787784846.025610:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:633bab11-ac8b-43e8-9332-f3266553ac37+13:30855:x1874627658223232:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 253us (357us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.025618:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1307 00000100:00100000:0.0:1787784846.027427:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.027482:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658223488 00000100:00100000:2.0:1787784846.027488:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658223488 00000100:00100000:2.0:1787784846.027521:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1308 00000100:00100000:2.0:1787784846.027530:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+13:30855:x1874627658223488:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784846.027594:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.027608:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.027617:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784846.027786:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003fe49b03 reply state 00000000961f5d1f transno 4294968165 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.027793:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784846.027853:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+15:30855:x1874627658223488:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 323us (428us total) trans 4294968165 rc 0/0 00000100:00100000:2.0:1787784846.027860:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1308 00010000:00010000:1.0:1787784846.028914:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.028925:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.028932:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.028939:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784846.028943:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.028951:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.028965:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784846.028968:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787784846.028972:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 2/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.028983:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f82 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.029190:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784846.029325:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658223616 00000100:00100000:3.0:1787784846.029332:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658223616 00000100:00100000:3.0:1787784846.029371:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1309 00000100:00100000:3.0:1787784846.029381:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+14:30855:x1874627658223616:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784846.029397:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784846.029412:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14eb expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.029450:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 15 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.029458:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 15 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.029463:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:3.0:1787784846.029465:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784846.029467:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784846.029499:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+15:30855:x1874627658223616:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 122us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784846.029504:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1309 00000100:00100000:0.0:1787784846.030599:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.030646:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658223744 00000100:00100000:1.0:1787784846.030652:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658223744 00000100:00100000:1.0:1787784846.030685:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 262 00000100:00100000:1.0:1787784846.030693:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+15:30855:x1874627658223744:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1787784846.030783:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+15:30855:x1874627658223744:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 90us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.030791:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 262 00000100:00100000:0.0:1787784846.032558:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784846.032698:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658224000 00000100:00100000:3.0:1787784846.032706:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658224000 00000100:00100000:3.0:1787784846.032848:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:3.0:1787784846.032859:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+15:30855:x1874627658224000:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784846.033102:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658224000:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 245us (551us total) trans 4294968166 rc 0/0 00000100:00100000:3.0:1787784846.033108:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 263 00000100:00100000:0.0:1787784846.035244:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.035440:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658224256 00000100:00100000:1.0:1787784846.035449:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658224256 00000100:00100000:1.0:1787784846.035502:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1310 00000100:00100000:1.0:1787784846.035513:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658224256:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784846.035800:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658224256:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 287us (561us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.035808:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1310 00000100:00100000:0.0:1787784846.040409:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.040556:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658224512 00000100:00100000:2.0:1787784846.040566:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658224512 00000100:00100000:2.0:1787784846.040635:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1311 00000100:00100000:2.0:1787784846.040646:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658224512:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784846.040667:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.040687:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f90 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14f2 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.040790:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.040806:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 2/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 17 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.040815:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.040847:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 17 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.040862:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 17 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.040875:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668430976:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784846.040956:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401800c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784846.046315:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784846.046445:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658224640 00000100:00100000:3.0:1787784846.046460:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658224640 00010000:00010000:3.0:1787784846.046502:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 18 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.046542:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 389 00000100:00100000:3.0:1787784846.046552:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895089b0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:633bab11-ac8b-43e8-9332-f3266553ac37+18:2356:x1874627658224640:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784846.046567:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784846.046573:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 18 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.046599:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 18 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.046628:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:3.0:1787784846.046634:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.046643:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.046650:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.046668:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784846.046718:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895089b0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:633bab11-ac8b-43e8-9332-f3266553ac37+18:2356:x1874627658224640:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 168us (412us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784846.046724:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 389 00010000:00010000:2.0:1787784846.046953:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc6f89 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14eb expref: 17 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.046966:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa5680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668430976:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784846.046986:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.046994:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047000:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784846.047018:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1:0x0]: returning size 0 00000004:00080000:2.0:1787784846.047347:0:20438:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1c:0x0], index 1: 0 00010000:00010000:2.0:1787784846.047470:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047479:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047485:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047491:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784846.047495:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047502:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047513:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784846.047516:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784846.047519:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047525:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc6f97 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047571:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f90 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14f2 expref: 17 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047581:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:2.0:1787784846.047583:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784846.047588:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc6f90 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14f2 expref: 17 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.047596:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2a00, rc 0) 00000100:00100000:2.0:1787784846.047640:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658224512:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 6996us (7238us total) trans 4294968167 rc 301/301 00000100:00100000:2.0:1787784846.047647:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1311 00000100:00100000:0.0:1787784846.053486:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.053587:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658224896 00000100:00100000:1.0:1787784846.053598:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658224896 00000100:00100000:1.0:1787784846.053654:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:1.0:1787784846.053664:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658224896:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784846.054595:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+17:30855:x1874627658224896:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 929us (1115us total) trans 4294968168 rc 0/0 00000100:00100000:1.0:1787784846.054605:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 264 00000100:00100000:3.0:1787784846.059101:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.059282:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658225024 00000100:00100000:2.0:1787784846.059292:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658225024 00000100:00100000:2.0:1787784846.059347:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1312 00000100:00100000:2.0:1787784846.059357:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+17:30855:x1874627658225024:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784846.059380:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.059401:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc6fa5 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b14f9 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.059468:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6fac lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.059482:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6fac lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.059493:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6fac lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784846.059523:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1:0x0]: returning size 0 00000004:00200000:2.0:1787784846.059527:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784846.059543:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6fac lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.059567:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905200/0xb0ce63137acc6fa5 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14f9 expref: 19 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.059593:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6fac lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14f9 expref: 18 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.059602:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784846.059612:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1787784846.059615:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989508905000, rc 0) 00000100:00100000:2.0:1787784846.059665:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+18:30855:x1874627658225024:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 308us (571us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.059672:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1312 00000100:00100000:0.0:1787784846.067655:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.067841:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658225152 00000100:00100000:1.0:1787784846.067852:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658225152 00000100:00080000:3.0:1787784846.067964:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784846.068496:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 361 00000100:00100000:1.0:1787784846.068507:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+5:2357:x1874627658225152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784846.068522:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784846.068537:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1500 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784846.068567:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784846.068583:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=637s v=(0 0) 00010000:00010000:1.0:1787784846.068588:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784846.068597:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 6 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.068616:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 6 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.068629:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98952a115400, rc 0) 00000100:00100000:1.0:1787784846.068679:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658225152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 176us (1034us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.068685:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 361 00000100:00100000:0.0:1787784846.071427:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.071476:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658225408 00000100:00100000:1.0:1787784846.071485:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658225408 00000100:00100000:1.0:1787784846.071534:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1313 00000100:00100000:1.0:1787784846.071541:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+18:30855:x1874627658225408:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784846.071638:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+18:30855:x1874627658225408:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 95us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.071646:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1313 00000100:00100000:3.0:1787784846.079384:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.079481:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658225536 00000100:00100000:0.0:1787784846.079490:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658225536 00000100:00100000:0.0:1787784846.079541:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 362 00000100:00100000:0.0:1787784846.079607:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:30855:x1874627658225536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784846.079623:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784846.079640:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1507 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.079671:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 7 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.079680:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784846.079684:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784846.079690:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 7 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.079927:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 7 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.079943:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 7 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.079958:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530505a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668431104:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784846.080078:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503b1b840 going to sleep for 11 seconds 00000100:00100000:3.0:1787784846.082779:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530505a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668431104:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784846.082819:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 7 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.082840:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00100000:3.0:1787784846.082898:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:633bab11-ac8b-43e8-9332-f3266553ac37+7:30855:x1874627658225536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3294us (3523us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784846.082907:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 362 00000100:00100000:0.0:1787784846.083820:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.083934:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658225664 00000100:00100000:1.0:1787784846.083942:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658225664 00010000:00010000:1.0:1787784846.083974:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 7 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.084048:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 390 00000100:00100000:1.0:1787784846.084057:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:633bab11-ac8b-43e8-9332-f3266553ac37+7:2357:x1874627658225664:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784846.084071:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784846.084090:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 7 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.084104:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 7 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.084127:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000065c4f3d2/0xb0ce63137acc6fb3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1500 expref: 7 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.084142:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784846.084197:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784846.084207:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 6 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.084217:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 6 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.084254:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 6 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.084268:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668431232:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784846.084330:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895054956c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784846.086558:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028b840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5795:1874627668431232:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784846.086582:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784846.086666:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658225664:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2609us (2847us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.086676:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 390 00000100:00100000:2.0:1787784846.111503:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.111623:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658225792 00000100:00100000:1.0:1787784846.111633:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658225792 00002000:00100000:1.0:1787784846.111683:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1c:0x0] (0->1048575) req@ffff989401fa5680 x1874627658225792/t0(0) o4->633bab11-ac8b-43e8-9332-f3266553ac37@192.168.203.43@tcp:727/0 lens 488/0 e 0 to 0 dl 1787784857 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784846.111695:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401fa5680 with x1874627658225792 ext(0->1048575) 00002000:00010000:1.0:1787784846.111701:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401fa5680 00000100:00100000:1.0:1787784846.111768:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 113 00000100:00100000:1.0:1787784846.111778:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:2356:x1874627658225792:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1787784846.125019:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:633bab11-ac8b-43e8-9332-f3266553ac37+8:2356:x1874627658225792:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 13236us (13518us total) trans 4294967376 rc 0/0 00000100:00100000:1.0:1787784846.125033:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 113 00002000:00100000:1.0:1787784846.125044:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1c:0x0] (0->1048575) req@ffff989401fa5680 x1874627658225792/t4294967376(0) o4->633bab11-ac8b-43e8-9332-f3266553ac37@192.168.203.43@tcp:727/0 lens 488/448 e 0 to 0 dl 1787784857 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784846.125054:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401fa5680 with x1874627658225792 ext(0->1048575) 00002000:00010000:1.0:1787784846.125060:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401fa5680 00000100:00100000:2.0:1787784846.128847:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.128905:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658225920 00000100:00100000:0.0:1787784846.128923:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658225920 00000100:00100000:0.0:1787784846.128982:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1314 00000100:00100000:0.0:1787784846.129004:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+18:30855:x1874627658225920:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1787784846.153105:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+18:30855:x1874627658225920:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 24106us (24255us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.153122:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1314 00000001:00080000:1.0:1787784846.153212:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968168 is committed 00000020:00080000:1.0:1787784846.153238:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 29 batchid updated 00000020:00080000:3.0:1787784846.153358:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 29 00000020:00080000:3.0:1787784846.153367:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 29 00000100:00100000:2.0:1787784846.155997:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784846.156222:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658226048 00000100:00100000:3.0:1787784846.156240:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658226048 00000100:00100000:3.0:1787784846.156294:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 363 00000100:00100000:3.0:1787784846.156303:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+8:2356:x1874627658226048:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:1.0:1787784846.202044:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967376 is committed 00000100:00100000:3.0:1787784846.202219:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+6:2356:x1874627658226048:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 45912us (46225us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784846.202231:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 363 00000100:00100000:0.0:1787784846.209725:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.209961:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658226304 00000100:00100000:1.0:1787784846.209971:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658226304 00000100:00100000:1.0:1787784846.210031:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1315 00000100:00100000:1.0:1787784846.210041:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658226304:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784846.210091:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784846.210117:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905000/0xb0ce63137acc6fac lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14f9 expref: 11 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.210137:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784846.210143:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784846.210235:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.210248:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.210258:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784846.211348:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b415d5fc reply state 00000000a5873acd transno 4294968169 ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.211361:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784846.211461:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+12:30855:x1874627658226304:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 1420us (1750us total) trans 4294968169 rc 0/0 00000100:00100000:1.0:1787784846.211473:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1315 00010000:00010000:0.0:1787784846.212782:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.212796:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.212804:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.212811:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784846.212815:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.212823:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.212838:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:0.0:1787784846.212841:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784846.212845:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 2/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.212852:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fc1 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.213668:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.213830:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658226432 00000100:00100000:1.0:1787784846.213840:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658226432 00000100:00100000:1.0:1787784846.213895:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1316 00000100:00100000:1.0:1787784846.213905:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658226432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784846.213927:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784846.213943:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b150e expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.213995:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 12 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.214008:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 12 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.214025:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784846.214028:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784846.214031:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98952a114a00, rc 0) 00000100:00100000:1.0:1787784846.214080:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+12:30855:x1874627658226432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 176us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.214087:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1316 00000100:00100000:2.0:1787784846.216447:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.216524:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658226560 00000100:00100000:1.0:1787784846.216532:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658226560 00000100:00100000:1.0:1787784846.216577:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 364 00000100:00100000:1.0:1787784846.216590:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+6:2356:x1874627658226560:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1787784846.217577:0:6670:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1787784846.217583:0:6670:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb mtime from disk: 0 -> 1787784836 00002000:00010000:1.0:1787784846.217585:0:6670:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb atime from disk: 0 -> 1787784836 00002000:00010000:1.0:1787784846.217587:0:6670:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb ctime from disk: 0 -> 1787784836 00002000:00010000:1.0:1787784846.217589:0:6670:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:1.0:1787784846.217645:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+7:2356:x1874627658226560:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 1056us (1203us total) trans 4294967377 rc 0/0 00000100:00100000:1.0:1787784846.217653:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 364 00000100:00100000:3.0:1787784846.219908:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.219998:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658226816 00000100:00100000:1.0:1787784846.220007:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658226816 00000100:00100000:1.0:1787784846.220064:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1317 00000100:00100000:1.0:1787784846.220073:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+12:30855:x1874627658226816:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1787784846.220130:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.220579:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 12 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.220593:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 4/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.220897:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 12 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.220913:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 12 pid: 5813 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.220928:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668431360:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784846.221004:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950631e180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784846.224007:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668431360:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784846.224033:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.225562:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.225614:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658226944 00000100:00100000:0.0:1787784846.225626:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658226944 00010000:00010000:0.0:1787784846.225662:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 13 pid: 5813 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.225695:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:0.0:1787784846.225704:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:633bab11-ac8b-43e8-9332-f3266553ac37+13:2357:x1874627658226944:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784846.225719:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784846.225726:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 13 pid: 5813 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.225745:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 13 pid: 5813 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.225765:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc6f7b lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14e4 expref: 13 pid: 5813 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.225782:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:0.0:1787784846.225787:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.225795:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.225803:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.225873:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784846.225923:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:633bab11-ac8b-43e8-9332-f3266553ac37+12:2357:x1874627658226944:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 219us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.225930:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 391 00010000:00010000:1.0:1787784846.226364:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226375:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226400:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226407:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226413:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226420:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784846.226424:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226431:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226440:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784846.226443:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787784846.226446:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 2/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.226452:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fcf lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.226512:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658226816:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 6440us (6611us total) trans 0 rc -22/-22 00000100:00100000:1.0:1787784846.226524:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1317 00000100:00100000:3.0:1787784846.228081:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.228625:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658227072 00000100:00100000:0.0:1787784846.228634:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658227072 00000100:00100000:2.0:1787784846.229009:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 365 00000100:00100000:2.0:1787784846.229019:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:633bab11-ac8b-43e8-9332-f3266553ac37+7:2356:x1874627658227072:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784846.229057:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.229075:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 7 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.229084:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784846.229088:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784846.229093:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.229121:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 7 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.229135:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 7 pid: 6669 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.229256:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668431488:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784846.229328:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895067c3780 going to sleep for 11 seconds 00000100:00100000:3.0:1787784846.235004:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668431488:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784846.235033:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.235948:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.236015:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658227200 00000100:00100000:0.0:1787784846.236022:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658227200 00010000:00010000:0.0:1787784846.236055:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 8 pid: 6669 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.236130:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 392 00000100:00100000:0.0:1787784846.236139:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:633bab11-ac8b-43e8-9332-f3266553ac37+8:2356:x1874627658227200:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784846.236195:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784846.236216:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 8 pid: 6669 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.236229:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 8 pid: 6669 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.236251:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eced4e3f/0xb0ce63137acc6fba lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1507 expref: 8 pid: 6669 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.236267:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784846.236271:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784846.236276:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.236286:0:21579:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.236314:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784846.236360:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:633bab11-ac8b-43e8-9332-f3266553ac37+7:2356:x1874627658227200:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 222us (415us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.236368:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 392 00010000:00010000:2.0:1787784846.236465:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236475:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236504:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236512:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236520:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236528:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784846.236532:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236541:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236552:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.236560:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6fd6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.236605:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:633bab11-ac8b-43e8-9332-f3266553ac37+6:2356:x1874627658227072:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 7588us (8529us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.236612:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 365 00000100:00100000:2.0:1787784846.239502:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.239559:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658227456 00000100:00100000:1.0:1787784846.239568:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658227456 00000100:00100000:1.0:1787784846.239616:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1318 00000100:00100000:1.0:1787784846.239625:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+11:30855:x1874627658227456:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784846.239689:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.239704:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.239715:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.239890:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.239900:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 2/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 12 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.239909:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.239936:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 12 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.239950:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 12 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.239964:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668431616:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784846.240034:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1a9300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784846.242665:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.242735:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658227584 00000100:00100000:0.0:1787784846.242750:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658227584 00010000:00010000:0.0:1787784846.242784:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 13 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.242823:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 393 00000100:00100000:0.0:1787784846.242831:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:633bab11-ac8b-43e8-9332-f3266553ac37+13:2356:x1874627658227584:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784846.242843:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784846.242849:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 13 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.242861:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 13 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.242881:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:0.0:1787784846.242886:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.242896:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.242903:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.242918:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784846.242967:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:633bab11-ac8b-43e8-9332-f3266553ac37+13:2356:x1874627658227584:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 137us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.242974:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 393 00010000:00010000:1.0:1787784846.243403:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6fc8 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b150e expref: 12 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.243420:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668431616:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784846.243438:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.243447:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.243453:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784846.243596:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968170 00000001:00080000:1.0:1787784846.243720:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968171 00000001:00080000:1.0:1787784846.243760:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968172 00000100:00100000:1.0:1787784846.243825:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668431744:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787784846.243863:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784846.243891:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784846.243936:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668431744 00000100:00100000:0.0:1787784846.243941:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668431744 00000100:00100000:0.0:1787784846.243972:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 63 00000100:00100000:0.0:1787784846.243980:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668431744:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787784846.244044:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668431744:12345-0@lo:1000:mdt00_001.0 Request processed in 65us (180us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.244051:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 63 00000100:00100000:1.0:1787784846.244201:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668431744:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1787784846.244470:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968173 00000100:00100000:1.0:1787784846.244511:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668431872:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787784846.244535:0:5812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784846.244557:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784846.244733:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668431872 00000100:00100000:3.0:1787784846.244737:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668431872 00000100:00100000:3.0:1787784846.244768:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 64 00000100:00100000:3.0:1787784846.244775:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895076cc850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5812:x1874627668431872:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787784846.244905:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895076cc850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5812:x1874627668431872:12345-0@lo:1000:mdt00_001.0 Request processed in 131us (368us total) trans 4294967378 rc 0/0 00000100:00100000:3.0:1787784846.244912:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 64 00000100:00100000:1.0:1787784846.245353:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5812:1874627668431872:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1787784846.245415:0:5812:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968174 00010000:00010000:1.0:1787784846.245438:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245447:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245455:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245461:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784846.245465:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245473:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245485:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784846.245488:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784846.245492:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245498:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115000/0xb0ce63137acc6fe4 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245522:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245529:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245535:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245541:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784846.245544:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245549:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245556:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:1.0:1787784846.245558:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784846.245564:0:5812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14b3 expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245575:0:5812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14b3 expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245583:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14b3 expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.245606:0:5812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2600/0xb0ce63137acc6eb0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14b3 expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.245617:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432000:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784846.245669:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784846.249476:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.249561:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432000:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784846.249586:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.249600:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc6ea9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.249628:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.249640:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 46 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.249648:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 4/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.249687:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 46 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.249702:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 46 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.249715:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432128:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784846.249784:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784846.249839:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658227712 00000100:00100000:0.0:1787784846.249845:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658227712 00000100:00100000:0.0:1787784846.249889:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 265 00000100:00100000:0.0:1787784846.249897:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30826:x1874627658227712:12345-192.168.203.43@tcp:49:cat.0 00000100:00100000:0.0:1787784846.249977:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30826:x1874627658227712:12345-192.168.203.43@tcp:49:cat.0 Request processed in 81us (509us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.249984:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 265 00000100:00100000:3.0:1787784846.253051:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.253191:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658227840 00000100:00100000:2.0:1787784846.253199:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658227840 00000100:00100000:2.0:1787784846.253251:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 366 00000100:00100000:2.0:1787784846.253260:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+13:30826:x1874627658227840:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784846.253275:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.253292:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1515 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784846.253315:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1c:0x0] initial LVB size: 1048576, mtime: 0x6a8f6e84, atime: 0x6a8f6e84, ctime: 0x6a8f6e84, blocks: 0x803 00010000:00010000:2.0:1787784846.253336:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=637s v=(0 0) 00010000:00010000:2.0:1787784846.253341:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784846.253350:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 14 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.253364:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 14 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.253378:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2a00, rc 0) 00000100:00100000:2.0:1787784846.253427:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:30826:x1874627658227840:12345-192.168.203.43@tcp:101:cat.0 Request processed in 168us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.253434:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 366 00000100:00080000:3.0:1787784846.253563:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784846.256883:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432128:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784846.256910:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.260520:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784846.261396:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784846.261435:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.261452:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658227968 00000100:00100000:0.0:1787784846.261464:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658227968 00010000:00010000:0.0:1787784846.261560:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 47 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.261600:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 394 00000100:00100000:0.0:1787784846.261610:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658227968:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784846.261624:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784846.261630:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 47 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.261644:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 47 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.261665:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114000/0xb0ce63137acc6ed3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14ba expref: 47 pid: 10850 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.261696:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:0.0:1787784846.261702:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.261711:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.261719:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.261735:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658228096 00000100:00100000:2.0:1787784846.261739:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658228096 00010000:00010000:0.0:1787784846.261756:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00100000:2.0:1787784846.261771:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1c:0x0] (0->1048575) req@ffff9894011efc00 x1874627658228096/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:727/0 lens 488/0 e 0 to 0 dl 1787784857 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784846.261786:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894011efc00 with x1874627658228096 ext(0->1048575) 00002000:00010000:2.0:1787784846.261791:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894011efc00 00000100:00100000:0.0:1787784846.261811:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658227968:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 203us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.261819:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 394 00010000:00010000:3.0:1787784846.261929:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261937:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261948:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261953:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261959:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261964:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a114a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784846.261968:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261976:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261985:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:3.0:1787784846.261987:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784846.261991:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.261996:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114a00/0xb0ce63137acc6feb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.262006:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.262010:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.262015:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.262019:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98952a115400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784846.262021:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.262026:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.262032:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:3.0:1787784846.262034:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784846.262037:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 2/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.262045:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115400/0xb0ce63137acc6fdd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784846.262374:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784846.262380:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00080000:1.0:1787784846.262383:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784846.262484:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 114 00000100:00100000:2.0:1787784846.262492:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2356:x1874627658228096:12345-192.168.203.43@tcp:3:cat.0 00000040:00080000:3.0:1787784846.265037:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:3.0:1787784846.265051:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784846.266139:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658227456:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 26507us (26636us total) trans 4294968175 rc 0/0 00000100:00100000:2.0:1787784846.266183:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1318 00000020:00080000:2.0:1787784846.266191:0:5812:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:0.0:1787784846.270958:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.271293:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658228352 00000100:00100000:1.0:1787784846.271304:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658228352 00000100:00100000:1.0:1787784846.271400:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 266 00000100:00100000:1.0:1787784846.271413:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+16:30855:x1874627658228352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1787784846.272406:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1787784846.272518:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968176 00010000:00010000:1.0:1787784846.272632:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.272649:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.272661:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.272670:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.272676:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.272683:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.272706:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787784846.272791:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:633bab11-ac8b-43e8-9332-f3266553ac37+18:30855:x1874627658228352:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1380us (1839us total) trans 4294968177 rc 0/0 00000100:00100000:1.0:1787784846.272800:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 266 00000020:00080000:1.0:1787784846.272805:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1787784846.272880:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784846.272890:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.272897:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784846.272901:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.272908:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.272920:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784846.272923:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787784846.272927:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.272934:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc6ff9 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.272950:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784846.272952:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784846.272953:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784846.287426:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2356:x1874627658228096:12345-192.168.203.43@tcp:3:cat.0 Request processed in 24927us (25982us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1787784846.287438:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 114 00002000:00100000:3.0:1787784846.287451:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1c:0x0] (0->1048575) req@ffff9894011efc00 x1874627658228096/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:727/0 lens 488/440 e 0 to 0 dl 1787784857 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784846.287461:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894011efc00 with x1874627658228096 ext(0->1048575) 00002000:00010000:3.0:1787784846.287466:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9894011efc00 00000100:00100000:0.0:1787784846.297685:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.297786:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658228480 00000100:00100000:2.0:1787784846.297795:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658228480 00002000:00100000:2.0:1787784846.297852:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1c:0x0] (1048576->1052671) req@ffff989403124000 x1874627658228480/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:727/0 lens 488/0 e 0 to 0 dl 1787784857 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784846.297862:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403124000 with x1874627658228480 ext(1048576->1052671) 00002000:00010000:2.0:1787784846.297867:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989403124000 00000100:00100000:2.0:1787784846.297896:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 115 00000100:00100000:2.0:1787784846.297906:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2354:x1874627658228480:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:2.0:1787784846.298499:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2354:x1874627658228480:12345-192.168.203.43@tcp:3:cat.0 Request processed in 593us (822us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.298507:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 115 00002000:00100000:2.0:1787784846.298513:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1c:0x0] (1048576->1052671) req@ffff989403124000 x1874627658228480/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:727/0 lens 488/440 e 0 to 0 dl 1787784857 ref 1 fl Complete:/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784846.298521:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403124000 with x1874627658228480 ext(1048576->1052671) 00002000:00010000:2.0:1787784846.298524:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989403124000 00000100:00100000:1.0:1787784846.302941:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.302995:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658228736 00000100:00100000:0.0:1787784846.303005:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658228736 00000100:00100000:0.0:1787784846.303057:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 267 00000100:00100000:0.0:1787784846.303068:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+99:30826:x1874627658228736:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784846.303344:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30826:x1874627658228736:12345-192.168.203.43@tcp:35:cat.0 Request processed in 277us (408us total) trans 4294968178 rc 0/0 00000100:00100000:0.0:1787784846.303352:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 267 00000100:00100000:3.0:1787784846.323545:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.323616:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658228864 00000100:00100000:2.0:1787784846.323626:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658228864 00000100:00100000:2.0:1787784846.323699:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1319 00000100:00100000:2.0:1787784846.323722:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:30856:x1874627658228864:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:2.0:1787784846.323748:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.323779:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7000 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b151c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.323846:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.323861:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.323873:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.323891:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.323899:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.323906:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324283:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324293:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324306:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784846.324336:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:2.0:1787784846.324341:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784846.324357:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324369:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324377:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324383:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324390:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324395:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784846.324399:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324406:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324416:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:2.0:1787784846.324419:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784846.324424:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324434:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7007 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324444:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324450:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324456:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324460:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784846.324464:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324469:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324475:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff989520131e00) 00010000:00010000:2.0:1787784846.324478:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784846.324481:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324486:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc700e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324514:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7000 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324533:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 101 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.324540:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:2.0:1787784846.324542:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784846.324545:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3a00, rc 0) 00000100:00100000:2.0:1787784846.324608:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:30856:x1874627658228864:12345-192.168.203.43@tcp:101:stat.0 Request processed in 890us (1072us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.324616:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1319 00000100:00100000:1.0:1787784846.336276:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.336548:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658228992 00000100:00100000:0.0:1787784846.336558:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658228992 00000100:00100000:0.0:1787784846.336614:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1320 00000100:00100000:0.0:1787784846.336624:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30857:x1874627658228992:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:0.0:1787784846.336643:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784846.336661:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc701c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1523 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.336728:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.336743:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.336759:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.336773:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.336781:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.336787:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337131:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337141:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337184:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784846.337218:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:0.0:1787784846.337223:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784846.337245:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337257:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337264:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337275:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337282:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337287:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784846.337291:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337298:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337308:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:0.0:1787784846.337311:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784846.337320:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337327:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7023 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337339:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337346:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337351:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337356:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784846.337359:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337365:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337371:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff989520108b00) 00010000:00010000:0.0:1787784846.337373:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784846.337376:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337381:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc702a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337414:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc701c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 48 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337427:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 47 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.337434:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:0.0:1787784846.337435:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784846.337634:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a800, rc 0) 00000100:00100000:0.0:1787784846.337699:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30857:x1874627658228992:12345-192.168.203.43@tcp:101:stat.0 Request processed in 1075us (1431us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.337707:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1320 00000100:00100000:1.0:1787784846.342918:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.343346:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658229120 00000100:00100000:0.0:1787784846.343356:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658229120 00000100:00100000:0.0:1787784846.343427:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 367 00000100:00100000:0.0:1787784846.343436:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658229120:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:0.0:1787784846.343454:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784846.343471:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b152a expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.343498:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784846.343503:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784846.343512:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.343529:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.343542:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150be00, rc 0) 00000100:00100000:0.0:1787784846.343603:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658229120:12345-192.168.203.43@tcp:101:stat.0 Request processed in 168us (694us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.343610:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 367 00000100:00080000:1.0:1787784846.344193:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784846.347328:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.350456:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658229376 00000100:00100000:2.0:1787784846.350469:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658229376 00000100:00100000:2.0:1787784846.350543:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1321 00000100:00100000:2.0:1787784846.350554:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:30573:x1874627658229376:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784846.350574:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.350595:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc703f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1531 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.350718:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.350739:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.350757:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 46 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.350770:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 5/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.350805:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 46 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.350818:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 46 pid: 20438 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.350832:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432256:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784846.350942:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 102 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.350951:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 102 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.350959:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432384:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784846.350983:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895067c3780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784846.354969:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432256:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784846.354983:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668432384:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784846.355009:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.355811:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.355994:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658229504 00000100:00100000:1.0:1787784846.356001:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658229504 00010000:00010000:1.0:1787784846.356037:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 47 pid: 20438 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.356077:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:1.0:1787784846.356086:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658229504:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784846.356101:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784846.356107:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 47 pid: 20438 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.356118:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 47 pid: 20438 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.356138:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7031 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1523 expref: 47 pid: 20438 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.356205:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:1.0:1787784846.356211:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.356411:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.356418:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784846.356471:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+46:2357:x1874627658229504:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 385us (662us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.356479:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:3.0:1787784846.357842:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.357923:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658229632 00000100:00100000:2.0:1787784846.357930:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658229632 00010000:00010000:2.0:1787784846.357958:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 103 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.357990:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:2.0:1787784846.357997:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c94850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:2356:x1874627658229632:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784846.358008:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784846.358013:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 103 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.358023:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 103 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.358040:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7015 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b151c expref: 103 pid: 5811 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.358061:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:2.0:1787784846.358065:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.358074:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.358081:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.358105:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784846.358189:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c94850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:2356:x1874627658229632:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 151us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784846.358201:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 396 00010000:00010000:1.0:1787784846.359091:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.359102:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784846.359121:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:1.0:1787784846.359459:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc703f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.359480:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.359489:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:1.0:1787784846.359492:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1787784846.359495:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:1.0:1787784846.359547:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30573:x1874627658229376:12345-192.168.203.43@tcp:101:bash.0 Request processed in 8995us (12227us total) trans 4294968179 rc 0/0 00000100:00100000:1.0:1787784846.359556:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1321 00000100:00100000:3.0:1787784846.367101:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.369469:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658229888 00000100:00100000:2.0:1787784846.369482:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658229888 00000100:00100000:2.0:1787784846.369552:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1322 00000100:00100000:2.0:1787784846.369563:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:30573:x1874627658229888:12345-192.168.203.43@tcp:36:bash.0 00010000:00010000:2.0:1787784846.370464:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.370485:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.370497:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784846.370548:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968180 00000004:00010000:2.0:1787784846.370575:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e99d0cd3 reply state 00000000aa9633e0 transno 4294968181 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.370585:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000001:00080000:2.0:1787784846.371867:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968181 00000100:00100000:2.0:1787784846.372377:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:30573:x1874627658229888:12345-192.168.203.43@tcp:36:bash.0 Request processed in 2810us (5280us total) trans 4294968182 rc 0/0 00000100:00100000:2.0:1787784846.372387:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1322 00000020:00080000:2.0:1787784846.372791:0:5811:0:(tgt_main.c:784:tgt_ses_key_exit()) total 3 transactions per RPC 00010000:00010000:1.0:1787784846.375278:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.375295:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.375303:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.375309:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784846.375314:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.375325:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.375340:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:1.0:1787784846.375343:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784846.375348:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.375354:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7054 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.375462:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784846.375957:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658230016 00000100:00100000:2.0:1787784846.375966:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658230016 00000100:00100000:2.0:1787784846.376018:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 368 00000100:00100000:2.0:1787784846.376028:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:30573:x1874627658230016:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784846.376044:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784846.376061:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d70fd6c9/0xb0ce63137acc705b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1538 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.376095:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 15 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.376104:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.376112:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784846.376116:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784846.376121:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d70fd6c9/0xb0ce63137acc705b lrc: 5/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1538 expref: 15 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.376186:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.376200:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 10 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.376214:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668432512:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784846.377176:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 15 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784846.377191:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 15 pid: 6671 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784846.377208:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668432640:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784846.377239:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989518a329c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784846.382312:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.382676:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658230144 00000100:00100000:1.0:1787784846.382684:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658230144 00010000:00010000:1.0:1787784846.382723:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 11 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.382764:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 397 00000100:00100000:1.0:1787784846.382774:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658230144:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784846.382788:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784846.382794:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 11 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.382808:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 11 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.382831:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784846.382886:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658230144:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 113us (582us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.382898:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 397 00010000:00010000:3.0:1787784846.383317:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b0669d52/0xb0ce63137acc7038 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b152a expref: 10 pid: 6670 timeout: 737 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.383335:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668432512:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784846.387717:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668432640:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784846.387987:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d70fd6c9/0xb0ce63137acc705b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1538 expref: 15 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.388009:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b2a00, rc 0) 00000100:00100000:3.0:1787784846.388072:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:30573:x1874627658230016:12345-192.168.203.43@tcp:101:bash.0 Request processed in 12044us (12612us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784846.388080:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 368 00000100:00100000:2.0:1787784846.390676:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.391850:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658230272 00000100:00100000:1.0:1787784846.391870:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658230272 00010000:00010000:1.0:1787784846.391929:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 15 pid: 6671 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.392260:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 398 00000100:00100000:1.0:1787784846.392279:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658230272:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784846.392301:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784846.392309:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 15 pid: 6671 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.392326:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 15 pid: 6671 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.392355:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8922d23/0xb0ce63137acc6ff2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1515 expref: 15 pid: 6671 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.392374:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784846.392380:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784846.392387:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d70fd6c9/0xb0ce63137acc705b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1538 expref: 14 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.392399:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d70fd6c9/0xb0ce63137acc705b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1538 expref: 14 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.392451:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d70fd6c9/0xb0ce63137acc705b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1538 expref: 14 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.392467:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668432768:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784846.392566:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784846.397129:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668432768:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784846.397190:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784846.397812:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658230272:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5536us (7138us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.397825:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 398 00000100:00100000:2.0:1787784846.397938:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.398011:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658230400 00000100:00100000:1.0:1787784846.398018:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658230400 00002000:00010000:1.0:1787784846.398051:0:6674:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/28 (0->18446744073709551615) 00002000:00100000:1.0:1787784846.398056:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304a940 with x1874627658230400 ext(0->18446744073709551615) 00002000:00010000:1.0:1787784846.398067:0:6674:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98940304a940. 00000100:00100000:1.0:1787784846.398098:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 116 00000100:00100000:1.0:1787784846.398107:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658230400:12345-192.168.203.43@tcp:10:bash.0 00002000:00010000:1.0:1787784846.398449:0:6674:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb size from disk: 1048576 -> 0 00002000:00010000:1.0:1787784846.398454:0:6674:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb mtime from disk: 1787784836 -> 1787784845 00002000:00010000:1.0:1787784846.398456:0:6674:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb atime from disk: 1787784836 -> 1787784836 00002000:00010000:1.0:1787784846.398459:0:6674:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb ctime from disk: 1787784836 -> 1787784845 00002000:00010000:1.0:1787784846.398461:0:6674:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb blocks from disk: 2051 -> 2051 00000100:00100000:1.0:1787784846.398506:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658230400:12345-192.168.203.43@tcp:10:bash.0 Request processed in 400us (568us total) trans 4294967379 rc 0/0 00000100:00100000:1.0:1787784846.398514:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 116 00002000:00010000:1.0:1787784846.398517:0:6674:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/28 (0->18446744073709551615) 00002000:00100000:1.0:1787784846.398520:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304a940 with x1874627658230400 ext(0->18446744073709551615) 00002000:00010000:1.0:1787784846.398525:0:6674:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98940304a940. 00000100:00100000:0.0:1787784846.409532:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.409708:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658230528 00000100:00100000:1.0:1787784846.409718:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658230528 00000100:00100000:1.0:1787784846.409939:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1323 00000100:00100000:1.0:1787784846.409950:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:30858:x1874627658230528:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:1.0:1787784846.409974:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784846.409995:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc7062 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b153f expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410059:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410073:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410085:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410098:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410106:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410113:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410439:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410452:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 2/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 106 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410460:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 4/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x48 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410490:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 106 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.410504:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 106 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784846.410517:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668432896:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784846.410603:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989527cce6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784846.413542:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.413624:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658230784 00000100:00100000:0.0:1787784846.413641:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658230784 00000100:00100000:1.0:1787784846.413650:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668432896:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784846.413679:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x48 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.413699:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 268 00000100:00100000:0.0:1787784846.413709:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:10750:x1874627658230784:12345-192.168.203.43@tcp:35:ldlm_bl.0 00000100:00100000:0.0:1787784846.414015:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:10750:x1874627658230784:12345-192.168.203.43@tcp:35:ldlm_bl.0 Request processed in 307us (480us total) trans 4294968183 rc 0/0 00000100:00100000:0.0:1787784846.414023:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 268 00000100:00100000:2.0:1787784846.416043:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784846.416181:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658230912 00000100:00100000:3.0:1787784846.416188:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658230912 00010000:00010000:3.0:1787784846.416323:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 108 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.416365:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 399 00000100:00100000:3.0:1787784846.416375:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:2357:x1874627658230912:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784846.416388:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784846.416394:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 108 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.416405:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 108 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.416425:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7046 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1531 expref: 108 pid: 5812 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.416442:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:3.0:1787784846.416447:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x48 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.416457:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.416464:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 4/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784846.416492:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784846.416531:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.416542:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2357:x1874627658230912:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 165us (503us total) trans 0 rc 0/0 00010000:00010000:0.0:1787784846.416544:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784846.416555:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 399 00000004:00200000:0.0:1787784846.416587:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 0 00000004:00200000:0.0:1787784846.416592:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784846.416622:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416635:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416645:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416652:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416659:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416665:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784846.416673:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416681:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416692:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:0.0:1787784846.416695:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784846.416699:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416707:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7069 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416717:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416721:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416725:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416728:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784846.416730:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416733:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416738:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff98950a38c300) 00010000:00010000:0.0:1787784846.416740:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784846.416742:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416745:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7070 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416771:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc7062 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b153f expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416783:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641200/0xb0ce63137acc7077 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b153f expref: 106 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.416788:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:0.0:1787784846.416790:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784846.416791:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641200, rc 0) 00000100:00100000:0.0:1787784846.416811:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:30858:x1874627658230528:12345-192.168.203.43@tcp:101:stat.0 Request processed in 6865us (7290us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784846.416816:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1323 00000100:00100000:3.0:1787784846.428898:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784846.429039:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231040 00000100:00100000:1.0:1787784846.429048:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231040 00000100:00100000:1.0:1787784846.429134:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1324 00000100:00100000:1.0:1787784846.429144:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+46:30859:x1874627658231040:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:1.0:1787784846.429209:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784846.429235:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc707e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1546 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429294:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429307:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429316:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429330:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429336:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429341:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429681:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc7093 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429690:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc7093 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429694:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc7093 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784846.429723:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 0 00000004:00200000:1.0:1787784846.429726:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784846.429808:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc7093 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429820:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc7093 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429825:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429832:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429838:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429842:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784846.429845:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429849:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429857:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:1.0:1787784846.429860:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784846.429864:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429869:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7085 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429878:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429882:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429886:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429890:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784846.429891:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429894:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429900:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff98950a38c300) 00010000:00010000:1.0:1787784846.429902:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784846.429905:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429908:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc708c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.429931:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc707e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1546 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.430053:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640200/0xb0ce63137acc7093 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1546 expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784846.430061:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff989520108d00) 00010000:00010000:1.0:1787784846.430063:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784846.430065:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640200, rc 0) 00000100:00100000:1.0:1787784846.430121:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:30859:x1874627658231040:12345-192.168.203.43@tcp:101:stat.0 Request processed in 978us (1230us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784846.430131:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1324 00000100:00100000:2.0:1787784846.436193:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784846.436231:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231168 00000100:00100000:0.0:1787784846.436243:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231168 00000100:00100000:0.0:1787784846.436309:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 369 00000100:00100000:0.0:1787784846.436322:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658231168:12345-192.168.203.43@tcp:101:stat.0 00010000:00010000:0.0:1787784846.436338:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784846.436358:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc709a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b154d expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784846.436419:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668433024:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784846.436624:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1840 going to sleep for 11 seconds 00000100:00100000:0.0:1787784846.442763:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668433024:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784846.442803:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc709a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b154d expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.442826:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc709a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b154d expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784846.442842:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a200, rc 0) 00000100:00100000:0.0:1787784846.442904:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658231168:12345-192.168.203.43@tcp:101:stat.0 Request processed in 6582us (6760us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784846.442918:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 369 00000100:00100000:3.0:1787784847.113234:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.113342:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.113379:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.113416:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784847.113477:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231808 00000100:00100000:1.0:1787784847.113486:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231808 00000100:00100000:1.0:1787784847.113540:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:1.0:1787784847.113550:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658231808:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784847.113623:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658231808:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 74us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784847.113631:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:3.0:1787784847.113805:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784847.113904:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231296 00000100:00100000:2.0:1787784847.113909:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231296 00000100:00100000:2.0:1787784847.113939:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 597 00000100:00100000:2.0:1787784847.113947:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:2357:x1874627658231296:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784847.113979:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:2357:x1874627658231296:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (755us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784847.113987:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 597 00000100:00100000:0.0:1787784847.114082:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231552 00000100:00100000:1.0:1787784847.114083:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658232064 00000100:00100000:1.0:1787784847.114088:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658232064 00000100:00100000:0.0:1787784847.114090:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231552 00000100:00100000:1.0:1787784847.114128:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 411 00000100:00100000:1.0:1787784847.114137:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658232064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784847.114220:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658232064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 84us (804us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784847.114226:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 411 00000100:00100000:0.0:1787784847.114289:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 412 00000100:00100000:0.0:1787784847.114300:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658231552:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784847.114339:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658231552:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (998us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784847.114346:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 412 00000100:00100000:3.0:1787784847.114794:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.114830:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.114847:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.114864:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.114887:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784847.114904:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231424 00000100:00100000:3.0:1787784847.114908:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231424 00000100:00100000:3.0:1787784847.114936:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658232192 00000100:00100000:3.0:1787784847.114939:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658232192 00000100:00100000:1.0:1787784847.114993:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658232320 00000100:00100000:1.0:1787784847.114997:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658232320 00000100:00100000:0.0:1787784847.115037:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231680 00000100:00100000:0.0:1787784847.115041:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231680 00000100:00100000:0.0:1787784847.115063:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658231936 00000100:00100000:0.0:1787784847.115066:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658231936 00000100:00080000:2.0:1787784847.115116:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784847.115121:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784847.115184:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658232448 00000100:00100000:2.0:1787784847.115189:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658232448 00000100:00100000:2.0:1787784847.115276:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 413 00000100:00100000:2.0:1787784847.115286:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658231680:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784847.115331:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658231680:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (492us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784847.115338:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 413 00000100:00100000:2.0:1787784847.115353:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 414 00000100:00100000:2.0:1787784847.115360:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658232448:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784847.115381:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+6:2357:x1874627658232448:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (494us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784847.115386:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 414 00000100:00100000:2.0:1787784847.115405:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:2.0:1787784847.115411:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658231424:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784847.115436:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658231424:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (645us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784847.115442:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:3.0:1787784847.122657:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 453 00000100:00100000:3.0:1787784847.122675:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+18:2357:x1874627658232192:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784847.122774:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:633bab11-ac8b-43e8-9332-f3266553ac37+18:2357:x1874627658232192:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 105us (7908us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784847.122782:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 453 00000100:00100000:1.0:1787784847.123116:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:1.0:1787784847.123127:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2357:x1874627658232320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784847.123192:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:633bab11-ac8b-43e8-9332-f3266553ac37+5:2357:x1874627658232320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 67us (9387us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784847.123200:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:0.0:1787784847.127433:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 416 00000100:00100000:0.0:1787784847.127449:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658231936:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784847.127547:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658231936:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 102us (12697us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784847.127555:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 416 00010000:00010000:2.0:1787784847.281384:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784847.882916:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967367 is committed 00010000:00010000:3.0:1787784848.305307:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784848.884904:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668433152:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784848.884985:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784848.885102:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668433152 00000100:00100000:0.0:1787784848.885110:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668433152 00000100:00100000:0.0:1787784848.885417:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1787784848.885426:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668433152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784848.885466:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784848.885530:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668433152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (543us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784848.885537:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1787784848.885586:0:3277:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967367 gen 2 00000004:00080000:0.0:1787784848.885591:0:3277:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401fa7840, transno 4294967367 00000100:00100000:0.0:1787784848.885604:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668433152:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787784848.889789:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787784849.330935:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784850.353698:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784850.933479:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784850.933511:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668433280:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784850.933611:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784850.933705:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668433280 00000100:00100000:3.0:1787784850.933715:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668433280 00000100:00100000:3.0:1787784850.933800:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1787784850.933814:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aa580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668433280:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784850.933860:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1787784850.933912:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aa580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668433280:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784850.933919:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00080000:3.0:1787784850.933942:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784850.933993:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668433280:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784850.935468:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784850.935497:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668433408:0@lo:400:kworker.0 00000100:00100000:2.0:1787784850.935516:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668433536:0@lo:400:kworker.0 00000100:00100000:3.0:1787784850.936139:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668433920:0@lo:400:kworker.0 00000100:00100000:2.0:1787784850.937092:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784850.937253:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784850.937264:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668433664:0@lo:400:kworker.0 00000100:00100000:2.0:1787784850.937300:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784850.937354:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668433536 00000100:00100000:2.0:1787784850.937364:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668433536 00000100:00100000:2.0:1787784850.937419:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668433664 00000100:00100000:2.0:1787784850.937423:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668433664 00000100:00100000:2.0:1787784850.937464:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1787784850.937475:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668433536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784850.937543:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668433536:12345-0@lo:400:kworker.0 Request processed in 69us (453us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784850.937550:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1787784850.937573:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1787784850.937580:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668433664:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784850.940662:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784850.940753:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668433792:0@lo:400:kworker.0 00000100:00100000:1.0:1787784850.940815:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784850.940879:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668433408 00000100:00100000:1.0:1787784850.940888:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668433408 00000100:00100000:1.0:1787784850.940944:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 598 00000100:00100000:1.0:1787784850.940954:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668433408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784850.941286:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668433664:12345-0@lo:400:kworker.0 Request processed in 3700us (3979us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784850.941301:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:0.0:1787784850.941679:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668433792 00000100:00100000:0.0:1787784850.941689:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668433792 00000100:00100000:0.0:1787784850.941750:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:0.0:1787784850.941760:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668433792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784850.942718:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668433792:12345-0@lo:400:kworker.0 Request processed in 954us (1897us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784850.942732:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:0.0:1787784850.942871:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668433792:0@lo:400:kworker.0 00000100:00100000:0.0:1787784850.942936:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668433664:0@lo:400:kworker.0 00000100:00100000:0.0:1787784850.942950:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668433536:0@lo:400:kworker.0 00000100:00100000:1.0:1787784850.943073:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668433408:12345-0@lo:400:kworker.0 Request processed in 2118us (2421us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784850.943082:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 598 00000100:00100000:1.0:1787784850.943141:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668433408:0@lo:400:kworker.0 00000100:00100000:3.0:1787784850.945140:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784850.945251:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668433920 00000100:00100000:3.0:1787784850.945261:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668433920 00000100:00100000:3.0:1787784850.945331:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1787784850.945344:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668433920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784850.945432:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668433920:12345-0@lo:400:kworker.0 Request processed in 91us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784850.945653:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00080000:3.0:1787784850.945687:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784850.945695:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784850.945730:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668433920:0@lo:400:kworker.0 00010000:00010000:3.0:1787784851.390684:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784851.558102:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784851.558186:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658232576 00000100:00100000:3.0:1787784851.558199:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658232576 00000100:00100000:3.0:1787784851.558274:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1325 00000100:00100000:3.0:1787784851.558286:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+18:30854:x1874627658232576:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1787784851.558422:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:633bab11-ac8b-43e8-9332-f3266553ac37+18:30854:x1874627658232576:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 137us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784851.558430:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1325 00000100:00080000:3.0:1787784851.558454:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784851.565965:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784851.568361:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658232832 00000100:00100000:1.0:1787784851.568382:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658232832 00000100:00100000:1.0:1787784851.568585:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 269 00000100:00100000:1.0:1787784851.568598:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+18:30854:x1874627658232832:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784851.569852:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:633bab11-ac8b-43e8-9332-f3266553ac37+19:30854:x1874627658232832:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1247us (3892us total) trans 4294968184 rc 0/0 00000100:00100000:1.0:1787784851.569866:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 269 00000001:00080000:1.0:1787784851.598467:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967379 is committed 00000001:00080000:1.0:1787784851.598490:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967377 is committed 00000001:00080000:1.0:1787784851.598510:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967378 is committed 00000001:00080000:1.0:1787784851.733456:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968183 is committed 00000001:00080000:2.0:1787784851.735255:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968177 is committed 00000020:00080000:2.0:1787784851.735278:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98952a1a9f00 refcount 2 committed 0 result 0 batchid 30 00000020:00080000:2.0:1787784851.735285:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895012e9e00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98952a1a9300 00000020:00080000:2.0:1787784851.735289:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895012e9500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895401a3000 00000020:00080000:3.0:1787784851.735868:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 30 committed batchid 29 00000020:00080000:3.0:1787784851.736172:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 29 00000100:00100000:1.0:1787784852.237951:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784852.238015:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784852.238357:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658232960 00000100:00100000:2.0:1787784852.238371:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658232960 00000100:00100000:2.0:1787784852.238460:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:2.0:1787784852.238481:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:2357:x1874627658232960:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784852.238585:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:2357:x1874627658232960:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 115us (649us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784852.238592:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:1.0:1787784852.239025:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784852.239070:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784852.239143:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233472 00000100:00100000:0.0:1787784852.239183:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233472 00000100:00100000:0.0:1787784852.239239:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 457 00000100:00100000:0.0:1787784852.239249:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658233472:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784852.239301:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658233472:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 54us (277us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784852.239309:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 457 00000100:00080000:2.0:1787784852.240026:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784852.251114:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784852.251492:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233216 00000100:00100000:3.0:1787784852.251506:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233216 00000100:00080000:2.0:1787784852.251619:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784852.254547:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233728 00000100:00100000:0.0:1787784852.254568:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233728 00000100:00100000:0.0:1787784852.254804:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 418 00000100:00100000:0.0:1787784852.254838:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658233216:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784852.255319:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658233216:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 500us (17299us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784852.255331:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 418 00000100:00100000:0.0:1787784852.255360:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 419 00000100:00100000:0.0:1787784852.255368:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658233728:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784852.255400:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658233728:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (16328us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784852.255407:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 419 00000100:00100000:1.0:1787784852.256691:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784852.256706:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233984 00000100:00100000:0.0:1787784852.256719:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233984 00000100:00100000:1.0:1787784852.256722:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233088 00000100:00100000:1.0:1787784852.256727:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233088 00000100:00080000:2.0:1787784852.257661:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784852.257669:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784852.257671:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784852.257904:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 458 00000100:00100000:0.0:1787784852.257910:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:1.0:1787784852.257922:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:2357:x1874627658233088:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784852.257924:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:633bab11-ac8b-43e8-9332-f3266553ac37+5:2357:x1874627658233984:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784852.258078:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:2357:x1874627658233088:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 162us (1397us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784852.258088:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 458 00000100:00100000:1.0:1787784852.258172:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784852.258410:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233344 00000100:00100000:3.0:1787784852.258419:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233344 00000100:00100000:0.0:1787784852.258473:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:633bab11-ac8b-43e8-9332-f3266553ac37+5:2357:x1874627658233984:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 552us (7373us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784852.258542:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 421 00000100:00100000:3.0:1787784852.258555:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658233344:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:2.0:1787784852.259609:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784852.259758:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784852.259782:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:2.0:1787784852.259808:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784852.259824:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233600 00000100:00100000:0.0:1787784852.259831:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233600 00000100:00100000:2.0:1787784852.259848:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784852.259879:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658233856 00000100:00100000:0.0:1787784852.259880:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658234112 00000100:00100000:2.0:1787784852.259886:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658233856 00000100:00100000:0.0:1787784852.259886:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658234112 00000100:00100000:0.0:1787784852.259908:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 422 00000100:00100000:0.0:1787784852.259919:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658233600:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784852.259934:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:2.0:1787784852.259954:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+10:2357:x1874627658233856:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784852.259978:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658233600:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784852.260000:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 422 00000100:00100000:2.0:1787784852.260002:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:633bab11-ac8b-43e8-9332-f3266553ac37+10:2357:x1874627658233856:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 58us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784852.260013:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:0.0:1787784852.260018:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 423 00000100:00100000:0.0:1787784852.260027:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:633bab11-ac8b-43e8-9332-f3266553ac37+5:2357:x1874627658234112:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:2.0:1787784852.260063:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784852.260068:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784852.260327:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:633bab11-ac8b-43e8-9332-f3266553ac37+5:2357:x1874627658234112:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 296us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784852.260334:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 423 00000100:00100000:2.0:1787784852.264093:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658233344:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 5383us (5762us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784852.264112:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 421 00000040:00080000:3.0:1787784852.401705:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784852.401727:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784852.402182:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 2 00000040:00080000:3.0:1787784852.402193:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784852.404549:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784853.425893:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784854.001436:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668434048:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784854.001505:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784854.001765:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434048 00000100:00100000:0.0:1787784854.001771:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434048 00000100:00080000:2.0:1787784854.001822:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784854.001871:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:0.0:1787784854.001882:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668434048:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784854.001926:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784854.001994:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668434048:12345-0@lo:13:osp-pre-0-0.0 Request processed in 113us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784854.002000:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1787784854.002091:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668434048:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787784854.450658:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784854.951402:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784854.951441:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784854.951523:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784854.951606:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658234240 00000100:00100000:3.0:1787784854.951616:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658234240 00000100:00100000:1.0:1787784854.951654:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658234368 00000100:00100000:1.0:1787784854.951661:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658234368 00000100:00100000:3.0:1787784854.951689:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658234496 00000100:00100000:3.0:1787784854.951695:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658234496 00000100:00100000:1.0:1787784854.951700:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 58 00000100:00100000:1.0:1787784854.951712:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+18:2355:x1874627658234240:12345-192.168.203.43@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787784854.951723:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 59 00010000:00010000:1.0:1787784854.951728:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787784854.951733:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+18:2355:x1874627658234368:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784854.951743:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784854.951766:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989500030600/0xb0ce63137acc6c72 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 6 type: PLN flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13d3 expref: 18 pid: 5803 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784854.951766:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff989500031000/0xb0ce63137acc6c6b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13a9 expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784854.951789:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784854.951843:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+17:2355:x1874627658234240:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 132us (451us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784854.951851:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 58 00000100:00100000:1.0:1787784854.951872:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 60 00000100:00100000:1.0:1787784854.951879:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+16:2354:x1874627658234496:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784854.951887:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784854.951897:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9894022fd800/0xb0ce63137acc6c79 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 8 type: PLN flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b13da expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784854.951901:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784854.951910:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784854.951932:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:2355:x1874627658234368:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 200us (488us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784854.951938:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+14:2354:x1874627658234496:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 57us (412us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784854.952540:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 59 00000100:00100000:3.0:1787784854.954935:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784854.955294:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658234624 00000100:00100000:2.0:1787784854.955304:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658234624 00000100:00100000:2.0:1787784854.955380:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 370 00000100:00080000:3.0:1787784854.955456:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784854.955486:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:633bab11-ac8b-43e8-9332-f3266553ac37+5:2353:x1874627658234624:12345-192.168.203.43@tcp:9:umount.0 00000020:00000080:2.0:1787784854.955523:0:13825:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xb0ce63137acc6c8e 00000100:00080000:2.0:1787784854.955533:0:13825:0:(import.c:65:import_set_state_nolock()) ffff98953aecc800 : changing import state from FULL to CLOSED 00000100:00100000:1.0:1787784854.955557:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 60 00000001:00000004:2.0:1787784854.955577:0:13825:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967379 00010000:00010000:0.0:1787784854.955583:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784854.955983:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:633bab11-ac8b-43e8-9332-f3266553ac37+2:2353:x1874627658234624:12345-192.168.203.43@tcp:9:umount.0 Request processed in 592us (1054us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784854.955991:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 370 00000100:00100000:0.0:1787784854.960062:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784854.960118:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658234752 00000100:00100000:1.0:1787784854.960128:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658234752 00000100:00100000:1.0:1787784854.960229:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 371 00000100:00100000:1.0:1787784854.960240:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+5:2353:x1874627658234752:12345-192.168.203.43@tcp:9:umount.0 00000020:00000080:1.0:1787784854.960269:0:6670:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xb0ce63137acc6c87 00000100:00080000:1.0:1787784854.960278:0:6670:0:(import.c:65:import_set_state_nolock()) ffff989540502800 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1787784854.960322:0:6670:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967367 00010000:00010000:2.0:1787784854.960356:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784854.960585:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:633bab11-ac8b-43e8-9332-f3266553ac37+2:2353:x1874627658234752:12345-192.168.203.43@tcp:9:umount.0 Request processed in 345us (542us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784854.960594:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 371 00000100:00080000:3.0:1787784854.960818:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784854.974383:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784854.974530:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658235392 00000100:00100000:0.0:1787784854.974540:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658235392 00000100:00100000:0.0:1787784854.974622:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1326 00000100:00100000:0.0:1787784854.974633:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+10:2353:x1874627658235392:12345-192.168.203.43@tcp:39:umount.0 00000020:00000080:0.0:1787784854.974838:0:5811:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xb0ce63137acc6c80 00000100:00080000:3.0:1787784854.974843:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000020:00010000:0.0:1787784854.974857:0:5811:0:(genops.c:1921:obd_stale_export_put()) Put export ffff989402f88000: total 0 00000100:00080000:0.0:1787784854.974864:0:5811:0:(import.c:65:import_set_state_nolock()) ffff98953f0b2000 : changing import state from FULL to CLOSED 00000020:00010000:3.0:1787784854.974891:0:5799:0:(genops.c:1907:obd_stale_export_get()) Get export ffff989402f88000: total 0 00000001:00000004:0.0:1787784854.974891:0:5811:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 1, last_transno is 4294968184 00010000:00010000:3.0:1787784854.974939:0:5799:0:(ldlm_lock.c:2651:ldlm_export_cancel_blocked_locks()) Export ffff989402f88000, canceled 0 locks, left on hash table 4. 00010000:00010000:3.0:1787784854.974979:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:3.0:1787784854.974983:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784854.975003:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc6cb8 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b140b expref: 9 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784854.975029:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff98950a38cf00) 00010000:00010000:3.0:1787784854.975032:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1787784854.975037:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e800/0xb0ce63137acc6f4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14d6 expref: 8 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784854.975064:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:3.0:1787784854.975067:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784854.975072:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc6ee1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b14c1 expref: 7 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784854.975087:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784854.975090:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784854.975109:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc6cd4 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1412 expref: 6 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784854.975121:0:5799:0:(ldlm_lock.c:2679:ldlm_export_cancel_locks()) Export ffff989402f88000, canceled 4 locks, left on hash table 0. 00010000:00010000:3.0:1787784854.975134:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784854.975136:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784854.975138:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784854.975632:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:633bab11-ac8b-43e8-9332-f3266553ac37+4:2353:x1874627658235392:12345-192.168.203.43@tcp:39:umount.0 Request processed in 999us (1262us total) trans 4294968185 rc 0/0 00000100:00100000:0.0:1787784854.975833:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1326 00010000:00010000:0.0:1787784855.473401:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784856.049805:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668434304:0@lo:400:kworker.0 00000100:00100000:0.0:1787784856.049882:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784856.049931:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668434816:0@lo:400:kworker.0 00000100:00100000:0.0:1787784856.049958:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784856.049986:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784856.049992:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668434432:0@lo:400:kworker.0 00000100:00100000:0.0:1787784856.050014:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784856.050051:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434304 00000100:00100000:0.0:1787784856.050058:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434304 00000100:00100000:0.0:1787784856.050109:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434432 00000100:00100000:0.0:1787784856.050112:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434432 00000100:00100000:0.0:1787784856.050143:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1787784856.050175:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668434304:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784856.050224:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668434304:12345-0@lo:400:kworker.0 Request processed in 51us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784856.050232:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1787784856.050253:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1787784856.050260:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668434432:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784856.050287:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668434432:12345-0@lo:400:kworker.0 Request processed in 27us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784856.050295:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00080000:0.0:1787784856.050314:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784856.050344:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668434432:0@lo:400:kworker.0 00000100:00100000:0.0:1787784856.050354:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668434304:0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050511:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b12d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668434560:0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050542:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784856.050576:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784856.050582:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668434176:0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050613:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784856.050636:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668434688:0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050658:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784856.050693:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434688 00000100:00100000:2.0:1787784856.050697:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434688 00000100:00100000:2.0:1787784856.050726:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:2.0:1787784856.050732:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668434688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050767:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668434688:12345-0@lo:400:kworker.0 Request processed in 35us (107us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784856.050773:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:2.0:1787784856.050792:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434176 00000100:00100000:2.0:1787784856.050795:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434176 00000100:00100000:2.0:1787784856.050817:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 600 00000100:00100000:2.0:1787784856.050823:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668434176:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050852:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668434176:12345-0@lo:400:kworker.0 Request processed in 29us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784856.050858:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 600 00000100:00100000:2.0:1787784856.050899:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668434176:0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050909:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668434688:0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.050931:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434816 00000100:00100000:2.0:1787784856.050934:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434816 00000100:00100000:2.0:1787784856.050961:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434560 00000100:00100000:2.0:1787784856.050964:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434560 00000100:00100000:2.0:1787784856.050981:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:2.0:1787784856.050987:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668434816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.051018:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668434816:12345-0@lo:400:kworker.0 Request processed in 32us (1060us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784856.051025:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:2.0:1787784856.051036:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:2.0:1787784856.051041:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668434560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784856.051069:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668434560:12345-0@lo:400:kworker.0 Request processed in 28us (526us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784856.051075:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:2.0:1787784856.051116:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b12d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668434560:0@lo:400:kworker.0 00000100:00080000:0.0:1787784856.052903:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784856.052911:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784856.052915:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784856.052965:0:3278:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967378 gen 2 00000004:00080000:0.0:1787784856.052972:0:3278:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967378 committed transno 4294967378 00000100:00100000:0.0:1787784856.052994:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668434816:0@lo:400:kworker.0 00000100:00100000:3.0:1787784856.053253:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668434944:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784856.053309:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784856.053396:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668434944 00000100:00100000:0.0:1787784856.053403:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668434944 00000100:00100000:0.0:1787784856.053447:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:0.0:1787784856.053457:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668434944:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784856.053491:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784856.053538:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668434944:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784856.053545:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:0.0:1787784856.053584:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668434944:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1787784856.502829:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1787784856.626851:0:7630:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 633bab11-ac8b-43e8-9332-f3266553ac37 00000020:00000080:2.0:1787784856.626860:0:7630:0:(genops.c:885:class_export_put()) final put ffff98953aec9800/633bab11-ac8b-43e8-9332-f3266553ac37 00000020:00000080:2.0:1787784856.626887:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff98953aec9800/633bab11-ac8b-43e8-9332-f3266553ac37 for lustre-OST0001 00000020:00000080:2.0:1787784856.626931:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff98953aecc800 for lustre-OST0001 00000001:00100000:2.0:1787784856.640222:0:5833:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 633bab11-ac8b-43e8-9332-f3266553ac37 00000001:00080000:2.0:1787784856.640245:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968185 is committed 00000020:00000080:2.0:1787784856.640255:0:5833:0:(genops.c:885:class_export_put()) final put ffff989402f88000/633bab11-ac8b-43e8-9332-f3266553ac37 00000020:00080000:2.0:1787784856.640277:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 30 batchid updated 00000020:00080000:2.0:1787784856.640294:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30 00000020:00080000:2.0:1787784856.640303:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30 00000020:00000080:1.0:1787784856.640357:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff989402f88000/633bab11-ac8b-43e8-9332-f3266553ac37 for lustre-MDT0000 00000020:00000080:1.0:1787784856.640425:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff98953f0b2000 for lustre-MDT0000 00000001:02000400:2.0:1787784856.656471:0:21775:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12r: lseek restores released file ===== 18:54:14 (1787784854) 00000100:00100000:2.0:1787784857.108005:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.108213:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658235520 00000100:00100000:0.0:1787784857.108223:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658235520 00000100:00100000:0.0:1787784857.108292:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1327 00000100:00100000:0.0:1787784857.108303:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:31491:x1874627658235520:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1787784857.108396:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:31491:x1874627658235520:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 95us (400us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784857.108403:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1327 00000100:00080000:2.0:1787784857.108529:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784857.149816:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.151180:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658235648 00000100:00100000:0.0:1787784857.151193:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658235648 00000100:00100000:0.0:1787784857.151280:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1328 00000100:00100000:0.0:1787784857.151291:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:31492:x1874627658235648:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784857.151308:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.151327:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70a1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1554 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151386:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151401:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151414:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151427:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151433:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151440:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787784857.151649:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784857.151684:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151696:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151702:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151708:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150be00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.151712:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151719:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151730:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784857.151733:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.151737:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151743:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70a8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151757:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151763:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151768:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151773:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.151776:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151782:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151788:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff989520108b00) 00010000:00010000:0.0:1787784857.151791:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.151794:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151800:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70af lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151834:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70a1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1554 expref: 101 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151843:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784857.151846:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.151849:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1554 expref: 101 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.151857:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00100000:0.0:1787784857.151926:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:31492:x1874627658235648:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 637us (2118us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784857.151933:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1328 00000100:00100000:3.0:1787784857.164054:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.164131:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658235904 00000100:00100000:0.0:1787784857.164141:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658235904 00000100:00100000:0.0:1787784857.164540:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1329 00000100:00100000:0.0:1787784857.164549:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:31492:x1874627658235904:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784857.164605:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784857.164641:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166e000/0xb0ce63137acc6d9f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1451 expref: 100 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.164670:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784857.164676:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784857.164902:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.164912:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.164918:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.164944:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 46 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.164956:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 46 pid: 5811 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.164965:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668435072:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784857.165041:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508949e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784857.167627:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.169256:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236032 00000100:00100000:1.0:1787784857.169270:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236032 00010000:00010000:1.0:1787784857.169317:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 47 pid: 5811 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784857.169375:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:1.0:1787784857.169495:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c96c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658236032:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784857.169512:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784857.169519:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 47 pid: 5811 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.169533:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 47 pid: 5811 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.169555:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:1.0:1787784857.169561:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.169570:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.169578:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.169596:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784857.169666:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c96c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:2357:x1874627658236032:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 282us (2047us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.169673:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 400 00010000:00010000:0.0:1787784857.172785:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc6d1a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1427 expref: 46 pid: 5811 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.172811:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668435072:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784857.172835:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.172846:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.172851:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.172878:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.172891:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.172900:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784857.177230:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 512 00010000:00010000:0.0:1787784857.177248:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.177268:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.177276:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.177282:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.177286:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.177295:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.177315:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784857.177319:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.177324:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.177331:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70b6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784857.177352:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006ae270dd reply state 00000000c9c9f710 transno 4294968186 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.177361:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784857.181477:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:31492:x1874627658235904:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 16918us (17422us total) trans 4294968186 rc 0/0 00000100:00100000:0.0:1787784857.181492:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1329 00010000:00010000:0.0:1787784857.185701:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.185717:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.185725:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.185731:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.185735:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.185743:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.185763:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff989520108b00) 00010000:00010000:0.0:1787784857.185766:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.185770:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.185776:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70bd lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784857.193874:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.193974:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236160 00000100:00100000:0.0:1787784857.193985:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236160 00000100:00100000:0.0:1787784857.194046:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1330 00000100:00100000:0.0:1787784857.194055:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+100:31493:x1874627658236160:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784857.194093:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.194119:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70c4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b155b expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.194221:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70cb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.194235:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.194245:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784857.194304:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784857.194308:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 32 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784857.194332:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.194361:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc70c4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b155b expref: 102 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.194379:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70cb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b155b expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.194398:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:0.0:1787784857.194401:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784857.194404:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00100000:0.0:1787784857.194466:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:31493:x1874627658236160:12345-192.168.203.43@tcp:101:bash.0 Request processed in 412us (602us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784857.194473:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1330 00000100:00100000:2.0:1787784857.196354:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.196474:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236288 00000100:00100000:1.0:1787784857.196482:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236288 00000100:00100000:1.0:1787784857.196527:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1331 00000100:00100000:1.0:1787784857.196536:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+101:31493:x1874627658236288:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784857.196553:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784857.196570:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc70d2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1562 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196623:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196636:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196645:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196657:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196663:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196670:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196762:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc70e7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196769:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc70e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196775:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc70e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784857.196803:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 512 00000004:00200000:1.0:1787784857.196806:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784857.196823:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc70e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196835:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196842:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196848:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196854:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784857.196858:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196865:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196875:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:1.0:1787784857.196877:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784857.196881:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196887:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc70d9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196898:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196908:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196914:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196919:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784857.196922:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196927:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196933:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff98950a38c300) 00010000:00010000:1.0:1787784857.196935:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784857.196937:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196942:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc70e0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196964:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc70d2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1562 expref: 103 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196974:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641a00/0xb0ce63137acc70e7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1562 expref: 102 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.196981:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784857.196983:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784857.196986:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641a00, rc 0) 00000100:00100000:1.0:1787784857.197041:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:31493:x1874627658236288:12345-192.168.203.43@tcp:101:bash.0 Request processed in 506us (694us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.197048:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1331 00000100:00100000:3.0:1787784857.198754:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.198831:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236416 00000100:00100000:0.0:1787784857.198840:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236416 00000100:00100000:0.0:1787784857.198891:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1332 00000100:00100000:0.0:1787784857.198900:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:31493:x1874627658236416:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784857.198918:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.198937:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70ee lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1569 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.198994:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199007:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199018:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199031:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199038:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199044:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199121:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199128:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199134:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199139:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.199371:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199380:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199455:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.199458:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.199463:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199469:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc70f5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199481:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199486:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199492:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199497:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.199499:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199505:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199511:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff989520108800) 00010000:00010000:0.0:1787784857.199514:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.199517:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199522:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc70fc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199556:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70ee lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1569 expref: 103 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199568:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.199570:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.199574:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc70ee lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1569 expref: 103 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.199583:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150be00, rc 0) 00000100:00100000:0.0:1787784857.199679:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:31493:x1874627658236416:12345-192.168.203.43@tcp:101:bash.0 Request processed in 778us (931us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784857.199687:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1332 00000100:00100000:3.0:1787784857.211801:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.211991:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236544 00000100:00100000:2.0:1787784857.212001:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236544 00000100:00100000:2.0:1787784857.212062:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1333 00000100:00100000:2.0:1787784857.212072:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+46:31494:x1874627658236544:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784857.212102:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784857.212122:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7103 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1570 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.212541:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.212556:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.212567:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784857.212629:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784857.212633:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 32 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784857.212665:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.212689:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7103 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 48 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.212713:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.212722:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784857.212725:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784857.212728:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2a00, rc 0) 00000100:00100000:2.0:1787784857.212828:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+47:31494:x1874627658236544:12345-192.168.203.43@tcp:101:rm.0 Request processed in 758us (1034us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784857.212836:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1333 00000100:00100000:2.0:1787784857.224453:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.224569:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236672 00000100:00100000:0.0:1787784857.224580:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236672 00000100:00100000:0.0:1787784857.224649:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1334 00000100:00100000:0.0:1787784857.224659:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+47:31494:x1874627658236672:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784857.224679:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.224701:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7111 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1577 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224766:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224780:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224792:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224805:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224812:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224818:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224921:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7126 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224930:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7126 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.224937:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7126 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784857.224966:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 512 00000004:00200000:0.0:1787784857.224969:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784857.224990:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7126 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225003:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225010:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225016:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225023:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.225027:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225034:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225045:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:0.0:1787784857.225049:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.225053:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225059:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7118 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225072:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225078:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225094:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225099:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.225102:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225106:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225113:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff989520108800) 00010000:00010000:0.0:1787784857.225115:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.225118:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225123:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc711f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225200:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7111 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1577 expref: 49 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225221:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a800/0xb0ce63137acc7126 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1577 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.225228:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.225231:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784857.225233:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a800, rc 0) 00000100:00100000:0.0:1787784857.225301:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31494:x1874627658236672:12345-192.168.203.43@tcp:101:rm.0 Request processed in 642us (856us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784857.225308:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1334 00000100:00100000:3.0:1787784857.230006:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.230102:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236800 00000100:00100000:0.0:1787784857.230112:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236800 00000100:00100000:0.0:1787784857.230213:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1335 00000100:00100000:0.0:1787784857.230224:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31494:x1874627658236800:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784857.230244:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.230436:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc712d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b157e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230506:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230521:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230533:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230547:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230555:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230562:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230624:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230632:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230649:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230656:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.230660:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230667:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230678:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.230682:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.230686:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230692:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7134 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230704:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230711:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230717:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230722:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.230735:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230742:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230750:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff989520108800) 00010000:00010000:0.0:1787784857.230753:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.230756:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230762:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc713b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230798:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc712d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b157e expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230807:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.230810:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.230814:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc712d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b157e expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.230828:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784857.230893:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31494:x1874627658236800:12345-192.168.203.43@tcp:101:rm.0 Request processed in 671us (896us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784857.230901:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1335 00000100:00100000:3.0:1787784857.234219:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.234660:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658236928 00000100:00100000:2.0:1787784857.234678:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658236928 00000100:00100000:2.0:1787784857.234740:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1336 00000100:00100000:2.0:1787784857.234751:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31494:x1874627658236928:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784857.234771:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784857.234790:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7142 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1585 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234855:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234869:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234881:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234899:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234905:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234911:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234968:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234974:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234980:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234985:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784857.234989:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.234995:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235006:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784857.235009:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784857.235013:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235018:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7149 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235031:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235036:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235041:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235044:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784857.235047:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235052:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235058:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff989520131500) 00010000:00010000:2.0:1787784857.235061:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784857.235063:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235068:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7150 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235108:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7142 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1585 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235117:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784857.235120:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784857.235123:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7142 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1585 expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.235132:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2400, rc 0) 00000100:00100000:2.0:1787784857.235213:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31494:x1874627658236928:12345-192.168.203.43@tcp:101:rm.0 Request processed in 463us (1002us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784857.235221:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1336 00000100:00100000:2.0:1787784857.255863:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.255961:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658237056 00000100:00100000:1.0:1787784857.255973:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658237056 00000100:00100000:1.0:1787784857.256032:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1337 00000100:00100000:1.0:1787784857.256042:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31495:x1874627658237056:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784857.256060:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784857.256092:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc7157 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b158c expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256206:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256222:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256233:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256246:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256252:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256264:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256325:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256334:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256340:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256347:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784857.256351:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256358:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256371:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784857.256374:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784857.256378:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256385:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc715e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256397:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256402:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256408:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256413:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784857.256416:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256422:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256429:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff98950a38c300) 00010000:00010000:1.0:1787784857.256432:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784857.256435:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256440:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7165 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256471:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc7157 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b158c expref: 49 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256482:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784857.256485:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784857.256488:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc7157 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b158c expref: 49 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.256497:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641c00, rc 0) 00000100:00100000:1.0:1787784857.256555:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31495:x1874627658237056:12345-192.168.203.43@tcp:101:cp.0 Request processed in 515us (702us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784857.256563:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1337 00000100:00100000:3.0:1787784857.262740:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.262878:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658237184 00000100:00100000:0.0:1787784857.262887:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658237184 00000100:00100000:0.0:1787784857.262944:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1338 00000100:00100000:0.0:1787784857.262955:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31495:x1874627658237184:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784857.262974:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.262994:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc716c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1593 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263059:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263074:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263100:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263116:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263123:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263130:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263231:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263239:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263245:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263251:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.263264:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263273:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263286:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.263289:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.263294:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263300:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7173 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263312:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263318:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263324:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263330:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.263333:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263339:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263346:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff989520108800) 00010000:00010000:0.0:1787784857.263349:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.263352:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263358:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc717a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263395:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc716c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1593 expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263405:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.263407:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.263411:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc716c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1593 expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.263421:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784857.263474:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31495:x1874627658237184:12345-192.168.203.43@tcp:101:cp.0 Request processed in 521us (743us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784857.263481:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1338 00000100:00100000:3.0:1787784857.268677:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.272233:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658237440 00000100:00100000:0.0:1787784857.272253:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658237440 00000100:00100000:0.0:1787784857.272370:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1339 00000100:00100000:0.0:1787784857.272380:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31495:x1874627658237440:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784857.272400:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.272420:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7181 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b159a expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.272546:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.272560:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.272571:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.272585:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.272592:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.272598:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784857.273607:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1a:0x0], index 0: 0 00010000:00010000:0.0:1787784857.274284:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274298:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274313:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274346:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274353:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274360:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274366:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.274371:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274378:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274390:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.274393:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.274397:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274404:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7188 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784857.274417:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000796aa928 reply state 00000000f9625ae7 transno 4294968187 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.274425:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784857.274458:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7181 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274472:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.274479:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:0.0:1787784857.274481:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784857.274483:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:0.0:1787784857.274558:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:31495:x1874627658237440:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2179us (5898us total) trans 4294968187 rc 0/0 00000100:00100000:0.0:1787784857.274567:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1339 00010000:00010000:3.0:1787784857.278505:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.278524:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.278531:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.278537:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784857.278542:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.278551:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.278567:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff989520108300) 00010000:00010000:3.0:1787784857.278571:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784857.278575:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.278581:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc718f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784857.279925:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.281007:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658237568 00000100:00100000:2.0:1787784857.281019:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658237568 00000100:00100000:2.0:1787784857.281101:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 372 00000100:00100000:2.0:1787784857.281111:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658237568:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784857.281128:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784857.281169:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15a1 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784857.281195:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784857.281214:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=648s v=(0 0) 00010000:00010000:2.0:1787784857.281218:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784857.281226:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.281245:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.281265:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:2.0:1787784857.281319:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658237568:12345-192.168.203.43@tcp:101:cp.0 Request processed in 210us (1399us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784857.281328:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 372 00000100:00080000:3.0:1787784857.281443:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784857.285458:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.285542:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658237696 00000100:00100000:2.0:1787784857.285552:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658237696 00000100:00100000:2.0:1787784857.285599:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 373 00000100:00100000:2.0:1787784857.285609:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:31495:x1874627658237696:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784857.285625:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784857.285653:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15a8 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.285692:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.285702:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784857.285706:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784857.285711:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.285745:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.285760:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 10 pid: 6670 timeout: 748 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784857.285773:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668435200:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784857.285859:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784857.289585:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.289697:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658237824 00000100:00100000:0.0:1787784857.289706:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658237824 00010000:00010000:0.0:1787784857.289746:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 11 pid: 6670 timeout: 748 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.289787:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:0.0:1787784857.289796:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658237824:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784857.289815:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784857.289838:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 11 pid: 6670 timeout: 748 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.289852:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 11 pid: 6670 timeout: 748 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.289875:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784857.289881:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784857.289889:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.289900:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.289937:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.289950:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668435328:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784857.290127:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a1 expref: 11 pid: 6670 timeout: 748 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784857.290141:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668435200:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784857.290188:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 10 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.290199:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403350200, rc 0) 00000100:00100000:0.0:1787784857.290760:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508949e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784857.292461:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:31495:x1874627658237696:12345-192.168.203.43@tcp:101:cp.0 Request processed in 6842us (7003us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784857.292473:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 373 00000100:00100000:3.0:1787784857.300668:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.300897:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238080 00000100:00100000:1.0:1787784857.300906:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238080 00000100:00100000:1.0:1787784857.300967:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 270 00000100:00100000:1.0:1787784857.300977:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:31495:x1874627658238080:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784857.301380:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:31495:x1874627658238080:12345-192.168.203.43@tcp:35:cp.0 Request processed in 402us (718us total) trans 4294968188 rc 0/0 00000100:00100000:1.0:1787784857.301387:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 270 00000100:00100000:0.0:1787784857.301495:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668435328:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784857.301519:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784857.301543:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658237824:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 11747us (11967us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784857.301551:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:1.0:1787784857.351460:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.351537:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238208 00000100:00100000:0.0:1787784857.351547:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238208 00000100:00100000:0.0:1787784857.351611:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1340 00000100:00100000:0.0:1787784857.351679:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:31498:x1874627658238208:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784857.351703:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784857.351724:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc71b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15af expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.351799:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.351812:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.351821:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.351837:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.351843:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.351850:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784857.352208:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.352252:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.352284:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.352310:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.352332:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238336 00000100:00100000:3.0:1787784857.352338:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238336 00000100:00100000:3.0:1787784857.352370:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 601 00000100:00100000:3.0:1787784857.352378:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658238336:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784857.352566:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238464 00000100:00100000:2.0:1787784857.352571:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238464 00000100:00100000:2.0:1787784857.352624:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238720 00000100:00100000:2.0:1787784857.352627:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238720 00000100:00100000:2.0:1787784857.352660:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238592 00000100:00100000:2.0:1787784857.352663:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238592 00000100:00100000:2.0:1787784857.352680:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 426 00000100:00100000:2.0:1787784857.352686:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658238464:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784857.352749:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658238464:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 62us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784857.352756:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 426 00000100:00100000:2.0:1787784857.352785:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 427 00000100:00100000:2.0:1787784857.352791:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658238720:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784857.352817:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658238720:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (566us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784857.352826:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 427 00000100:00100000:2.0:1787784857.352837:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 428 00000100:00100000:2.0:1787784857.352843:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658238592:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784857.352865:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:2357:x1874627658238592:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (554us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784857.352871:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 428 00000100:00080000:1.0:1787784857.353174:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784857.353178:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784857.354315:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658238336:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1934us (2026us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784857.354325:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 601 00010000:00010000:0.0:1787784857.354416:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354431:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354443:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784857.354481:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00000004:00200000:0.0:1787784857.354485:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784857.354504:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354516:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354531:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354537:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354544:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354551:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.354556:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354563:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354575:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784857.354578:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.354582:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354589:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc71b9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354603:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354610:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354616:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354622:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150be00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784857.354625:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354631:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354640:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff989520108800) 00010000:00010000:0.0:1787784857.354642:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784857.354645:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354651:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc71c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354690:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc71b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 104 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354706:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 103 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.354714:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:0.0:1787784857.354717:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784857.354719:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784857.354747:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:31498:x1874627658238208:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3128us (3296us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784857.354755:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1340 00000100:00100000:3.0:1787784857.360986:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.361071:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238848 00000100:00100000:2.0:1787784857.361095:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238848 00000100:00100000:2.0:1787784857.361188:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:2.0:1787784857.361199:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658238848:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784857.361226:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784857.361244:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bd332f0/0xb0ce63137acc71ce lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15b6 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784857.361300:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668435456:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784857.361379:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d600 going to sleep for 11 seconds 00002000:00010000:2.0:1787784857.368865:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784857.368875:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb mtime: 0 -> 1787784856 00002000:00010000:2.0:1787784857.368878:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb atime: 0 -> 1787784856 00002000:00010000:2.0:1787784857.368881:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb ctime: 0 -> 1787784856 00000100:00100000:2.0:1787784857.368906:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668435456:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784857.368941:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005bd332f0/0xb0ce63137acc71ce lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15b6 expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.368973:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bd332f0/0xb0ce63137acc71ce lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15b6 expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.368989:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351400, rc 0) 00000100:00100000:2.0:1787784857.369050:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658238848:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7852us (8072us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784857.369059:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 374 00000100:00100000:3.0:1787784857.376191:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.376370:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658238976 00000100:00100000:2.0:1787784857.376389:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658238976 00000100:00100000:2.0:1787784857.376447:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1341 00000100:00100000:2.0:1787784857.376458:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:31498:x1874627658238976:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784857.376478:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784857.376497:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc71d5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15bd expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.376570:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.376584:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.376596:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784857.376674:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 512 00000004:00200000:2.0:1787784857.376678:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784857.376700:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.376725:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc71d5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.376743:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 104 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.376751:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:2.0:1787784857.376754:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784857.376757:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351800, rc 0) 00000100:00100000:2.0:1787784857.376817:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:31498:x1874627658238976:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 360us (671us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784857.376824:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1341 00000100:00100000:3.0:1787784857.380999:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.383539:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658239104 00000100:00100000:1.0:1787784857.383553:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658239104 00000100:00100000:1.0:1787784857.383657:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 375 00000100:00100000:1.0:1787784857.383668:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658239104:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784857.383685:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784857.383700:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc71e3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15c4 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784857.383758:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668435584:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784857.383843:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953056b840 going to sleep for 11 seconds 00000100:00100000:1.0:1787784857.387487:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668435584:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784857.387525:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc71e3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15c4 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.387552:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc71e3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15c4 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.387565:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784857.387852:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658239104:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4184us (6861us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784857.387862:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 375 00000100:00100000:3.0:1787784857.393071:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.395939:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658239360 00000100:00100000:0.0:1787784857.395953:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658239360 00000100:00100000:0.0:1787784857.396015:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1342 00000100:00100000:0.0:1787784857.396025:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:31498:x1874627658239360:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:0.0:1787784857.396379:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].39, 136 off13496 00000100:00100000:0.0:1787784857.396598:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:31498:x1874627658239360:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 573us (3534us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784857.396607:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1342 00010000:00010000:1.0:1787784857.522388:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784857.522620:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784857.522629:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784857.522678:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 40 count 3 00000040:00080000:3.0:1787784857.522686:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784857.523050:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668435712:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784857.523231:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784857.528644:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.528687:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658239488 00000100:00100000:3.0:1787784857.528695:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658239488 00000100:00100000:3.0:1787784857.528745:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1343 00000100:00100000:3.0:1787784857.528755:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:31508:x1874627658239488:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784857.528790:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784857.528883:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:31508:x1874627658239488:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 130us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784857.528891:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1343 00000100:00100000:1.0:1787784857.529056:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668435712:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784857.532747:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.532839:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658239744 00000100:00100000:2.0:1787784857.532849:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658239744 00000100:00100000:2.0:1787784857.532905:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1344 00000100:00100000:2.0:1787784857.532916:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:31508:x1874627658239744:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784857.532937:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784857.532956:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc71ea lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15cb expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.533049:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc71f1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.533064:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc71f1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.533090:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc71f1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784857.533102:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:2.0:1787784857.533236:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc71ea lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15cb expref: 54 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.533253:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc71f1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15cb expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.533260:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:2.0:1787784857.533263:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784857.533267:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403350c00, rc 0) 00000100:00100000:2.0:1787784857.533325:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31508:x1874627658239744:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 412us (586us total) trans 4294968189 rc 0/0 00000100:00100000:2.0:1787784857.533333:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1344 00000100:00100000:3.0:1787784857.539570:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.539643:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.540658:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658239872 00000100:00100000:2.0:1787784857.540669:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658239872 00000100:00100000:2.0:1787784857.540721:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 376 00000100:00100000:2.0:1787784857.540732:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658239872:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784857.540773:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.540794:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.540821:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784857.540825:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784857.540830:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.540863:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 9 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.540876:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 9 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784857.540889:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a90b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668435840:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784857.540967:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784857.541239:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658240000 00000100:00100000:1.0:1787784857.541244:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658240000 00000100:00100000:1.0:1787784857.541283:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 61 00000100:00100000:1.0:1787784857.541290:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2355:x1874627658240000:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784857.541303:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784857.541334:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc71f1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15cb expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.541350:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:1.0:1787784857.541355:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784857.541381:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:2355:x1874627658240000:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 91us (1738us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.541388:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 61 00000100:00100000:1.0:1787784857.543203:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a90b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668435840:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784857.543226:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784857.543731:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.543782:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658240128 00000100:00100000:1.0:1787784857.543788:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658240128 00002000:00100000:1.0:1787784857.543827:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1a:0x0] (0->157) req@ffff989518f0bc00 x1874627658240128/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:738/0 lens 648/0 e 0 to 0 dl 1787784868 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784857.543845:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f0bc00 with x1874627658240128 ext(0->157) 00010000:00010000:1.0:1787784857.543852:0:18437:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 10 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.543866:0:18437:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 10 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784857.543875:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989518f0bc00 00000100:00100000:1.0:1787784857.543904:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 14 00000100:00100000:1.0:1787784857.543914:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658240128:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784857.544381:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658240128:12345-192.168.203.43@tcp:4:cp.0 Request processed in 468us (1249us total) trans 4294967368 rc 0/0 00000100:00100000:1.0:1787784857.544390:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 14 00002000:00100000:1.0:1787784857.544397:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1a:0x0] (0->157) req@ffff989518f0bc00 x1874627658240128/t4294967368(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:738/0 lens 648/448 e 0 to 0 dl 1787784868 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784857.544406:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f0bc00 with x1874627658240128 ext(0->157) 00010000:00010000:1.0:1787784857.544411:0:18437:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 12 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.544424:0:18437:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 12 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784857.544432:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989518f0bc00 00000100:00100000:2.0:1787784857.547362:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.547468:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658240256 00000100:00100000:1.0:1787784857.547476:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658240256 00010000:00010000:1.0:1787784857.547590:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 12 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784857.547627:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 402 00000100:00100000:1.0:1787784857.547637:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658240256:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784857.547649:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784857.547668:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 12 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.547683:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 12 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.547716:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc71ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15a8 expref: 12 pid: 13825 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.547731:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784857.547735:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784857.547741:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.547752:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.547815:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784857.547857:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658240256:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 222us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.547867:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 402 00010000:00010000:2.0:1787784857.550226:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784857.550245:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.550340:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.550350:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.550358:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.550365:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784857.550369:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.550378:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.550395:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.550402:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc71ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784857.550508:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658239872:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 9767us (10931us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.550522:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 376 00000100:00100000:2.0:1787784857.554408:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.554483:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658240512 00000100:00100000:3.0:1787784857.555000:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658240512 00000100:00100000:3.0:1787784857.555061:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1345 00000100:00100000:3.0:1787784857.555071:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:31508:x1874627658240512:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784857.555377:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:31508:x1874627658240512:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 306us (975us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784857.555385:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1345 00000100:00100000:2.0:1787784857.565579:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.566926:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658240640 00000100:00100000:1.0:1787784857.566940:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658240640 00000100:00100000:1.0:1787784857.567052:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 377 00000100:00100000:1.0:1787784857.567065:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658240640:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784857.567097:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784857.567119:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15d2 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784857.567170:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1a:0x0] initial LVB size: 158, mtime: 0x6a8f6e98, atime: 0x0, ctime: 0x6a8f6e98, blocks: 0x1 00010000:00010000:1.0:1787784857.567189:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=648s v=(0 0) 00010000:00010000:1.0:1787784857.567193:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784857.567204:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.567221:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.567235:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641400, rc 0) 00000100:00100000:1.0:1787784857.567301:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658240640:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 239us (1730us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.567309:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 377 00000100:00100000:1.0:1787784857.570946:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.571114:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658240896 00000100:00100000:3.0:1787784857.571123:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658240896 00000100:00100000:3.0:1787784857.571205:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1346 00000100:00100000:3.0:1787784857.571215:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+52:31508:x1874627658240896:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784857.571321:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+52:31508:x1874627658240896:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 107us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784857.571329:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1346 00000100:00100000:3.0:1787784857.574726:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784857.575032:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658241024 00000100:00100000:0.0:1787784857.575042:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658241024 00002000:00100000:0.0:1787784857.575177:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x1a:0x0] (0->4095) req@ffff989518f0a1c0 x1874627658241024/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:738/0 lens 488/0 e 0 to 0 dl 1787784868 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784857.575188:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f0a1c0 with x1874627658241024 ext(0->4095) 00002000:00010000:0.0:1787784857.575194:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989518f0a1c0 00000100:00100000:0.0:1787784857.575295:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 117 00000100:00100000:0.0:1787784857.575307:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658241024:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1787784857.575515:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658241024:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 206us (792us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784857.575523:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 117 00002000:00100000:0.0:1787784857.575531:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x1a:0x0] (0->4095) req@ffff989518f0a1c0 x1874627658241024/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:738/0 lens 488/4536 e 0 to 0 dl 1787784868 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784857.575539:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f0a1c0 with x1874627658241024 ext(0->4095) 00002000:00010000:0.0:1787784857.575543:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989518f0a1c0 00000100:00100000:0.0:1787784857.579103:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.579248:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658241152 00000100:00100000:1.0:1787784857.579269:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658241152 00000100:00100000:1.0:1787784857.579334:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1347 00000100:00100000:1.0:1787784857.579345:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:31508:x1874627658241152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784857.579365:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784857.579386:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc720d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15d9 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.579433:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.579447:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.579459:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.579708:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc720d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 54 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.579727:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.579736:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:1.0:1787784857.579739:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784857.579742:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641e00, rc 0) 00000100:00100000:1.0:1787784857.579799:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31508:x1874627658241152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 456us (723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.579807:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1347 00000100:00100000:1.0:1787784857.585990:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.586190:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658241280 00000100:00100000:3.0:1787784857.586199:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658241280 00000100:00100000:3.0:1787784857.586252:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1348 00000100:00100000:3.0:1787784857.586262:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31508:x1874627658241280:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784857.586295:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784857.586394:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31508:x1874627658241280:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 134us (411us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784857.586402:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1348 00000100:00100000:3.0:1787784857.591257:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.591320:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658241408 00000100:00100000:1.0:1787784857.591330:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658241408 00000100:00100000:1.0:1787784857.591385:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 378 00000100:00100000:1.0:1787784857.591404:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658241408:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784857.591444:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591462:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784857.591466:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784857.591472:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591487:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591515:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591524:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591533:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591540:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784857.591544:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591557:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591571:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784857.591578:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a4ce41d/0xb0ce63137acc721b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784857.591650:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658241408:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 247us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784857.591657:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 378 00000100:00100000:0.0:1787784857.595315:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784857.595385:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658241664 00000100:00100000:3.0:1787784857.595393:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658241664 00000100:00100000:3.0:1787784857.595452:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1349 00000100:00100000:3.0:1787784857.595462:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31508:x1874627658241664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784857.595530:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.595548:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.595556:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.595589:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.595603:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 53 pid: 20438 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784857.595617:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668435968:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784857.595690:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066779c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784857.597516:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784857.597543:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668435968:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784857.597567:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.597631:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658241792 00000100:00100000:0.0:1787784857.597637:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658241792 00010000:00010000:0.0:1787784857.597665:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 54 pid: 20438 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784857.597702:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 403 00000100:00100000:0.0:1787784857.597711:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658241792:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784857.597722:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784857.597728:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 54 pid: 20438 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.597741:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 54 pid: 20438 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.597760:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7214 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d9 expref: 54 pid: 20438 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.597773:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:0.0:1787784857.597778:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.597786:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.597859:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784857.597889:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784857.597927:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2357:x1874627658241792:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 218us (418us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784857.597938:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 403 00010000:00010000:3.0:1787784857.598092:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598100:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598343:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598351:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598358:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598364:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784857.598369:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598377:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598387:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:3.0:1787784857.598390:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784857.598393:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784857.598399:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7222 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784857.598516:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31508:x1874627658241664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 3054us (3206us total) trans 4294968190 rc 0/0 00000100:00100000:3.0:1787784857.598524:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1349 00000040:00080000:0.0:1787784857.599337:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784857.599342:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784857.600276:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784857.600287:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784857.604056:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784857.604380:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658242048 00000100:00100000:1.0:1787784857.604438:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658242048 00000100:00100000:1.0:1787784857.604491:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 271 00000100:00100000:1.0:1787784857.604501:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31508:x1874627658242048:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784857.604726:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:31508:x1874627658242048:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 226us (676us total) trans 4294968191 rc 0/0 00000100:00100000:1.0:1787784857.604735:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 271 00000040:00080000:2.0:1787784858.134804:0:21824:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784858.134814:0:21824:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784858.134858:0:21824:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 40 count 3 00000040:00080000:2.0:1787784858.134863:0:21824:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784858.186687:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.187037:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658242176 00000100:00100000:1.0:1787784858.187337:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658242176 00000100:00100000:1.0:1787784858.187416:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 379 00000100:00100000:1.0:1787784858.187426:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658242176:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784858.187444:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784858.187461:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15e0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.187487:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1787784858.187492:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784858.187748:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.187776:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.187789:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641600, rc 0) 00000100:00100000:1.0:1787784858.188409:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658242176:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 746us (1502us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.188421:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 379 00000001:00080000:3.0:1787784858.196323:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967368 is committed 00000001:00100000:3.0:1787784858.196339:0:6696:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 633bab11-ac8b-43e8-9332-f3266553ac37 00000020:00000080:3.0:1787784858.196343:0:6696:0:(genops.c:885:class_export_put()) final put ffff989540500000/633bab11-ac8b-43e8-9332-f3266553ac37 00000020:00000080:3.0:1787784858.196365:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff989540500000/633bab11-ac8b-43e8-9332-f3266553ac37 for lustre-OST0000 00000020:00000080:3.0:1787784858.196408:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff989540502800 for lustre-OST0000 00000100:00100000:1.0:1787784858.203471:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.203584:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658242432 00000100:00100000:0.0:1787784858.203594:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658242432 00000100:00100000:0.0:1787784858.203682:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1350 00000100:00100000:0.0:1787784858.203692:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:31509:x1874627658242432:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784858.203711:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784858.203730:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7230 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15e7 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.203825:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7237 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.203839:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7237 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.203851:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7237 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784858.203861:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:0.0:1787784858.203974:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7230 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e7 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.203991:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7237 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e7 expref: 106 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.204005:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:0.0:1787784858.204008:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784858.204010:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b800, rc 0) 00000100:00100000:0.0:1787784858.204064:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:31509:x1874627658242432:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 374us (602us total) trans 4294968192 rc 0/0 00000100:00100000:0.0:1787784858.204086:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1350 00000100:00100000:2.0:1787784858.208071:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.208300:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658242560 00000100:00100000:1.0:1787784858.208309:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658242560 00000100:00100000:1.0:1787784858.208361:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 380 00000100:00100000:1.0:1787784858.208370:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658242560:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:1.0:1787784858.208413:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.208433:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 8 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.208443:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.208452:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784858.208456:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784858.208460:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.208493:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.208507:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 17 pid: 13825 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.208522:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668436096:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784858.208623:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 8 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.208633:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 8 pid: 6670 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.208643:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668436224:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784858.208666:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d1480 going to sleep for 11 seconds 00000100:00100000:0.0:1787784858.219900:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.219939:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.219987:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658242688 00000100:00100000:1.0:1787784858.219996:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658242688 00010000:00010000:1.0:1787784858.220036:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 18 pid: 13825 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784858.220091:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658242816 00000100:00100000:3.0:1787784858.220096:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658242816 00000100:00100000:1.0:1787784858.220099:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00100000:1.0:1787784858.220111:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2357:x1874627658242688:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784858.220120:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 9 pid: 6670 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220127:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1787784858.220185:0:5796:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:1.0:1787784858.220193:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 18 pid: 13825 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220206:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 18 pid: 13825 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220231:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784858.220281:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2357:x1874627658242688:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 171us (393us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.220290:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00100000:3.0:1787784858.220300:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:3.0:1787784858.220313:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658242816:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784858.220322:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784858.220332:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 9 pid: 6670 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.220344:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 9 pid: 6670 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.220359:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784858.220364:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784858.220370:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.220380:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.220410:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784858.220418:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc7229 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e0 expref: 17 pid: 13825 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.220431:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668436096:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784858.220445:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc7206 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15d2 expref: 9 pid: 6670 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784858.220454:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658242816:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 142us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.220458:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668436224:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784858.220468:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 405 00010000:00010000:1.0:1787784858.220487:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220498:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220505:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220526:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220533:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220541:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220548:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784858.220551:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220560:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220571:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.220583:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc7245 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.220623:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658242560:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 12255us (12560us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.220630:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 380 00000100:00100000:2.0:1787784858.227246:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.227497:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658243072 00000100:00100000:1.0:1787784858.227506:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658243072 00000100:00100000:1.0:1787784858.227549:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 272 00000100:00100000:1.0:1787784858.227558:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:31509:x1874627658243072:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:1.0:1787784858.227618:0:5814:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x28:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7237 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e7 expref: 106 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784858.227841:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968193 00000001:00080000:1.0:1787784858.229647:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968194 00010000:00010000:1.0:1787784858.229804:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.229824:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.229833:0:5814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 109 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.229841:0:5814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 5/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.229880:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 109 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.229896:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 109 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.229910:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668436352:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784858.230008:0:5814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.230019:0:5814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 53 pid: 5812 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.230028:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668436480:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784858.230052:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9c180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784858.232636:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668436352:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784858.232651:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5814:1874627668436480:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784858.232691:0:5814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.233466:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.233541:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.233594:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658243200 00000100:00100000:1.0:1787784858.233601:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658243200 00010000:00010000:1.0:1787784858.233631:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 110 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.233761:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658243328 00000100:00100000:1.0:1787784858.233765:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658243328 00010000:00010000:1.0:1787784858.233780:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 54 pid: 5812 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.233802:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:1.0:1787784858.233811:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c96450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:2357:x1874627658243200:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784858.233826:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784858.233832:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 110 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.233845:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 110 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.233866:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc71c7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15af expref: 110 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.233881:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:1.0:1787784858.233886:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.233895:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.233902:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784858.233945:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c96450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:2357:x1874627658243200:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 135us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.233956:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:1.0:1787784858.233996:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 407 00000100:00100000:1.0:1787784858.234003:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658243328:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784858.234011:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784858.234017:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 54 pid: 5812 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.234026:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 54 pid: 5812 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.234038:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7196 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b159a expref: 54 pid: 5812 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.234048:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:1.0:1787784858.234052:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.234059:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.234066:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.234102:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784858.234121:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2356:x1874627658243328:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 117us (580us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.234128:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 407 00010000:00010000:1.0:1787784858.234226:0:5814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.234234:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784858.234337:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968195 00000001:00080000:1.0:1787784858.234522:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968196 00000001:00080000:1.0:1787784858.234576:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968197 00000100:00100000:1.0:1787784858.234644:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668436608:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:1.0:1787784858.236014:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784858.236101:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787784858.236324:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668436608 00000100:00100000:3.0:1787784858.236331:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668436608 00000100:00100000:3.0:1787784858.236414:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 65 00000100:00100000:3.0:1787784858.236424:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668436608:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1787784858.236499:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668436608:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 77us (488us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784858.236506:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 65 00000100:00080000:1.0:1787784858.236572:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784858.236604:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668436608:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:1.0:1787784858.237273:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968198 00000100:00100000:1.0:1787784858.237346:0:5814:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668436736:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:1.0:1787784858.237383:0:5814:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784858.237447:0:5814:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066d240 going to sleep for 11 seconds 00000100:00100000:3.0:1787784858.237481:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668436736 00000100:00100000:3.0:1787784858.237486:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668436736 00000100:00100000:3.0:1787784858.237547:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:3.0:1787784858.237554:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5814:x1874627668436736:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1787784858.237891:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5814:x1874627668436736:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 336us (506us total) trans 4294967369 rc 0/0 00000100:00100000:3.0:1787784858.237898:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00000100:00100000:1.0:1787784858.238019:0:5814:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288f00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5814:1874627668436736:0@lo:1000:mdt_rdpg00_000.0 00010000:00010000:1.0:1787784858.238040:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.238049:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.238056:0:5814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.238062:0:5814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784858.238066:0:5814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.238091:0:5814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.238106:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:1.0:1787784858.238108:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:1.0:1787784858.238114:0:5814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.238120:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc724c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787784858.238392:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x4e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1787784858.238484:0:5814:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].12, 64 off8984 00000004:00080000:1.0:1787784858.238494:0:5814:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 649325346141ns, next at 650325344588ns, rc = 0 00000001:00080000:1.0:1787784858.238501:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968199 00010000:00010000:1.0:1787784858.238533:0:5814:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520108c00) 00010000:00010000:1.0:1787784858.238536:0:5814:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1787784858.238542:0:5814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7237 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15e7 expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.238683:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:31509:x1874627658243072:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 11125us (11442us total) trans 4294968200 rc 0/0 00000100:00100000:1.0:1787784858.238692:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 272 00000020:00080000:1.0:1787784858.238696:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1787784858.272621:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.272694:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658243584 00000100:00100000:3.0:1787784858.272704:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658243584 00000100:00100000:3.0:1787784858.272765:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1351 00000100:00100000:3.0:1787784858.272775:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:31510:x1874627658243584:12345-192.168.203.43@tcp:101:lseek_test.0 00010000:00010000:3.0:1787784858.272800:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784858.272822:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7253 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15ee expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.272896:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.272910:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.272922:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.272940:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.272948:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.272955:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273167:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7268 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273178:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7268 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273186:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7268 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784858.273197:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:3.0:1787784858.273291:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273300:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273307:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273314:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784858.273318:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273329:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273342:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:3.0:1787784858.273345:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784858.273350:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273356:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc725a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273368:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273374:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273380:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273385:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784858.273388:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273394:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273401:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff9895198b5000) 00010000:00010000:3.0:1787784858.273404:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784858.273407:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273415:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7261 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273473:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc7253 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15ee expref: 115 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273487:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7268 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15ee expref: 114 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.273494:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff9895198b5100) 00010000:00010000:3.0:1787784858.273496:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784858.273498:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784858.273560:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:31510:x1874627658243584:12345-192.168.203.43@tcp:101:lseek_test.0 Request processed in 786us (947us total) trans 4294968201 rc 0/0 00000100:00100000:3.0:1787784858.273568:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1351 00000100:00100000:1.0:1787784858.283333:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.283368:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658243840 00000100:00100000:1.0:1787784858.283377:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658243840 00000100:00100000:1.0:1787784858.283431:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 62 00000100:00100000:1.0:1787784858.283440:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:31510:x1874627658243840:12345-192.168.203.43@tcp:103:lseek_test.0 00010000:00010000:1.0:1787784858.283456:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784858.283481:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7268 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15ee expref: 114 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.283500:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff9895198b5100) 00010000:00010000:1.0:1787784858.283511:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784858.283557:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:31510:x1874627658243840:12345-192.168.203.43@tcp:103:lseek_test.0 Request processed in 118us (234us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.283564:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 62 00000100:00100000:1.0:1787784858.289843:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.290002:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658243968 00000100:00100000:3.0:1787784858.290010:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658243968 00000100:00100000:3.0:1787784858.290060:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1352 00000100:00100000:3.0:1787784858.290071:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:31510:x1874627658243968:12345-192.168.203.43@tcp:58:lseek_test.0 00010000:00010000:2.0:1787784858.290323:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.290337:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.290346:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784858.290571:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].40, 136 off13632 00000100:00100000:2.0:1787784858.290726:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:31510:x1874627658243968:12345-192.168.203.43@tcp:58:lseek_test.0 Request processed in 656us (890us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.290733:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1352 00000100:00100000:2.0:1787784858.294753:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.295090:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658244096 00000100:00100000:0.0:1787784858.295101:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658244096 00000100:00100000:0.0:1787784858.295188:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1353 00000100:00100000:0.0:1787784858.295197:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:31510:x1874627658244096:12345-192.168.203.43@tcp:101:lseek_test.0 00010000:00010000:0.0:1787784858.295221:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784858.295241:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15f5 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.295301:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.295312:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 4/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15f5 expref: 114 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.295331:0:20438:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.295348:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 3/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15f5 expref: 114 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.295361:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520131400) 00010000:00010000:0.0:1787784858.295364:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784858.295369:0:20438:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 3/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15f5 expref: 114 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.295376:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a000, rc 0) 00000100:00100000:0.0:1787784858.295437:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:31510:x1874627658244096:12345-192.168.203.43@tcp:101:lseek_test.0 Request processed in 241us (693us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784858.295446:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1353 00010000:00010000:0.0:1787784858.548142:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784858.609638:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784858.609645:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784858.609683:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 41 count 4 00000040:00080000:1.0:1787784858.609689:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784858.609801:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402187840 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668436864:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784858.609915:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066de40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784858.617627:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.617702:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402187840 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668436864:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784858.618265:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658244224 00000100:00100000:2.0:1787784858.618273:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658244224 00000100:00100000:2.0:1787784858.618334:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1354 00000100:00100000:2.0:1787784858.618345:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31512:x1874627658244224:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784858.618520:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784858.618617:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31512:x1874627658244224:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 273us (996us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.618626:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1354 00000100:00100000:0.0:1787784858.622024:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784858.622881:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658244352 00000100:00100000:2.0:1787784858.622892:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658244352 00000100:00100000:2.0:1787784858.622944:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1355 00000100:00100000:2.0:1787784858.622953:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31512:x1874627658244352:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1787784858.623015:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623032:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623044:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623057:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623064:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623084:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623107:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623115:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623121:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784858.623172:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00000004:00200000:2.0:1787784858.623177:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784858.623188:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623198:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623205:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623211:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623217:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784858.623221:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623228:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623239:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784858.623252:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784858.623256:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623263:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc7284 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623274:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623285:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623291:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623295:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784858.623298:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623304:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623312:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xefe2aa85 (ffff989520131c00) 00010000:00010000:2.0:1787784858.623314:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784858.623316:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623322:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc728b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623334:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623339:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623346:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623350:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784858.623353:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623360:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623366:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520131400) 00010000:00010000:2.0:1787784858.623368:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784858.623370:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.623376:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7292 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784858.623523:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31512:x1874627658244352:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 569us (1506us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.623531:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1355 00000100:00100000:3.0:1787784858.627314:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784858.627395:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658244608 00000100:00100000:2.0:1787784858.627404:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658244608 00000100:00100000:2.0:1787784858.627485:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1356 00000100:00100000:2.0:1787784858.627495:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:31512:x1874627658244608:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784858.627513:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784858.627533:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7299 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b15fc expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627622:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627648:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627670:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784858.627681:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:2.0:1787784858.627763:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627770:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627778:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627783:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784858.627787:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627795:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627806:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520131400) 00010000:00010000:2.0:1787784858.627809:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784858.627813:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627819:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72a0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627853:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7299 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15fc expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627868:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:2.0:1787784858.627870:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784858.627874:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7299 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15fc expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.627883:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2400, rc 0) 00000100:00100000:2.0:1787784858.627935:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:31512:x1874627658244608:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 441us (1310us total) trans 4294968202 rc 301/301 00000100:00100000:2.0:1787784858.627941:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1356 00000100:00100000:3.0:1787784858.630942:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.632086:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658244736 00000100:00100000:1.0:1787784858.632095:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658244736 00000100:00100000:1.0:1787784858.632205:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1357 00000100:00100000:1.0:1787784858.632215:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:31512:x1874627658244736:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784858.632417:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784858.632507:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:31512:x1874627658244736:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 293us (1571us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.632514:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1357 00000100:00100000:0.0:1787784858.635326:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.635506:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658244992 00000100:00100000:1.0:1787784858.635514:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658244992 00000100:00100000:1.0:1787784858.635562:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1358 00000100:00100000:1.0:1787784858.635571:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:31512:x1874627658244992:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784858.635588:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784858.635607:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72ae lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1603 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.636316:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc72b5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.636331:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc72b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.636342:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc72b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784858.636352:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x29:0x0]: returning size 0 00010000:00010000:1.0:1787784858.636390:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72ae lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1603 expref: 57 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.636406:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc72b5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1603 expref: 56 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.636415:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989503a6d200) 00010000:00010000:1.0:1787784858.636418:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784858.636421:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641400, rc 0) 00000100:00100000:1.0:1787784858.636470:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+56:31512:x1874627658244992:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 901us (1153us total) trans 4294968203 rc 0/0 00000100:00100000:1.0:1787784858.636481:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1358 00000100:00100000:0.0:1787784858.640035:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.640292:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658245248 00000100:00100000:3.0:1787784858.640301:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658245248 00000100:00100000:3.0:1787784858.640376:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1359 00000100:00100000:3.0:1787784858.640385:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+56:31512:x1874627658245248:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1787784858.640439:0:5811:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784858.640463:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc72b5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1603 expref: 56 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.640481:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989503a6d200) 00010000:00010000:3.0:1787784858.640490:0:5811:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784858.640511:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.640523:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 113 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.640532:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.640562:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 113 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.640574:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 113 pid: 10850 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784858.640589:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668436992:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784858.640668:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784858.642708:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668436992:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784858.642732:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.643547:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.643607:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658245376 00000100:00100000:1.0:1787784858.643614:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658245376 00010000:00010000:1.0:1787784858.643647:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 114 pid: 10850 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.643695:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 408 00000100:00100000:1.0:1787784858.643703:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:2356:x1874627658245376:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784858.643717:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784858.643723:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 114 pid: 10850 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.643734:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 114 pid: 10850 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.643775:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc71dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15bd expref: 114 pid: 10850 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.643791:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784858.643797:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.643805:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.643813:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.643841:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784858.643881:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:2356:x1874627658245376:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 179us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.643888:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 408 00010000:00010000:0.0:1787784858.643974:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.643981:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644200:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644223:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644236:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644366:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644375:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644382:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644389:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784858.644393:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644402:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644415:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:0.0:1787784858.644418:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784858.644423:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644431:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc72c3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644447:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644453:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644460:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644465:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150be00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784858.644469:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644475:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644482:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0xbd3ba9b (ffff989520108800) 00010000:00010000:0.0:1787784858.644484:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784858.644488:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.644507:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc72ca lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0xbd3ba9b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.644822:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+55:31512:x1874627658245248:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 4432us (4789us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784858.644832:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1359 00000100:00100000:3.0:1787784858.652031:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784858.656121:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658245504 00000100:00100000:2.0:1787784858.656134:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658245504 00000100:00100000:2.0:1787784858.656260:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1360 00000100:00100000:2.0:1787784858.656272:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:31512:x1874627658245504:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784858.656295:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784858.656322:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc72d1 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b160a expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.656371:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.656388:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.656399:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.656512:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc72d1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 57 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.656533:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 56 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.656541:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:2.0:1787784858.656544:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784858.656548:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:2.0:1787784858.656623:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+56:31512:x1874627658245504:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 353us (4602us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.656641:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1360 00000100:00100000:0.0:1787784858.661996:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.663004:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658245760 00000100:00100000:1.0:1787784858.663014:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658245760 00000100:00100000:1.0:1787784858.663187:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1361 00000100:00100000:1.0:1787784858.663196:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+56:31512:x1874627658245760:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784858.663273:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.663289:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.663300:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784858.663471:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d58e75ab reply state 0000000077d76d8b transno 4294968204 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.663480:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784858.663580:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+58:31512:x1874627658245760:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 385us (1591us total) trans 4294968204 rc 0/0 00000100:00100000:1.0:1787784858.663587:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1361 00010000:00010000:0.0:1787784858.665169:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.665181:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.665189:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.665195:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784858.665199:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.665207:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.665223:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:0.0:1787784858.665226:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787784858.665230:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 2/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.665236:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc72df lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.665752:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.665815:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658245888 00000100:00100000:3.0:1787784858.665822:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658245888 00000100:00100000:3.0:1787784858.665899:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1362 00000100:00100000:3.0:1787784858.665908:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+57:31512:x1874627658245888:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784858.665928:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784858.665943:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1611 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.665989:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 58 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.666002:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 58 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.666010:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:3.0:1787784858.666013:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784858.666016:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784858.666063:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+58:31512:x1874627658245888:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 157us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784858.666070:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1362 00000100:00100000:3.0:1787784858.668142:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.668225:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658246016 00000100:00100000:1.0:1787784858.668234:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658246016 00000100:00100000:1.0:1787784858.668285:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 273 00000100:00100000:1.0:1787784858.668294:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+58:31512:x1874627658246016:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1787784858.668395:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+58:31512:x1874627658246016:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 101us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.668402:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 273 00000100:00100000:2.0:1787784858.670903:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.671177:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658246272 00000100:00100000:1.0:1787784858.671186:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658246272 00000100:00100000:1.0:1787784858.671237:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 274 00000100:00100000:1.0:1787784858.671246:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+58:31512:x1874627658246272:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784858.671821:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+59:31512:x1874627658246272:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 573us (923us total) trans 4294968205 rc 0/0 00000100:00100000:1.0:1787784858.671830:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 274 00000100:00100000:0.0:1787784858.673567:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784858.673810:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658246528 00000100:00100000:2.0:1787784858.673818:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658246528 00000100:00100000:2.0:1787784858.673909:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1363 00000100:00100000:2.0:1787784858.673918:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+59:31512:x1874627658246528:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784858.674184:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+59:31512:x1874627658246528:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 267us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.674192:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1363 00000100:00100000:3.0:1787784858.679528:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.680294:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658246784 00000100:00100000:1.0:1787784858.680305:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658246784 00000100:00100000:1.0:1787784858.680391:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1364 00000100:00100000:1.0:1787784858.680401:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+59:31512:x1874627658246784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784858.680420:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784858.680439:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc72ed lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1618 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.680594:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.680610:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 60 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.680618:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.680653:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 60 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.680676:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 60 pid: 5811 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.680688:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402187840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668437120:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784858.680806:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954066de40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784858.685267:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.685695:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658246912 00000100:00100000:0.0:1787784858.685703:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658246912 00010000:00010000:0.0:1787784858.685738:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 61 pid: 5811 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.685776:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 409 00000100:00100000:0.0:1787784858.685785:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+61:2356:x1874627658246912:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784858.685799:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784858.685972:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 61 pid: 5811 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.685985:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 61 pid: 5811 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.686006:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:0.0:1787784858.686011:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.686021:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.686029:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.686047:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784858.686139:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+61:2356:x1874627658246912:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 355us (880us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784858.686173:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 409 00010000:00010000:1.0:1787784858.686382:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc72e6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1611 expref: 60 pid: 5811 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.686398:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402187840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668437120:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784858.686415:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686424:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686430:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784858.686447:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x29:0x0]: returning size 0 00000004:00080000:1.0:1787784858.686745:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1d:0x0], index 1: 0 00010000:00010000:1.0:1787784858.686934:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686943:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686955:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686962:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784858.686966:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686974:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686984:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:1.0:1787784858.686987:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784858.686990:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.686996:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641c00/0xb0ce63137acc72f4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.690513:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc72ed lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1618 expref: 60 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.690532:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff98950a38cd00) 00010000:00010000:1.0:1787784858.690536:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784858.690541:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc72ed lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1618 expref: 60 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.690556:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640400, rc 0) 00000100:00100000:1.0:1787784858.690628:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+59:31512:x1874627658246784:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 10228us (11106us total) trans 4294968206 rc 301/301 00000100:00100000:1.0:1787784858.690640:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1364 00000100:00100000:2.0:1787784858.693555:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.693673:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658247168 00000100:00100000:3.0:1787784858.693682:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658247168 00000100:00100000:3.0:1787784858.693724:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 275 00000100:00100000:3.0:1787784858.693733:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+59:31512:x1874627658247168:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784858.694037:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+60:31512:x1874627658247168:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 304us (489us total) trans 4294968207 rc 0/0 00000100:00100000:3.0:1787784858.694044:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 275 00000100:00100000:2.0:1787784858.696552:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.696666:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658247296 00000100:00100000:0.0:1787784858.696674:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658247296 00000100:00100000:0.0:1787784858.696724:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1365 00000100:00100000:0.0:1787784858.696733:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+60:31512:x1874627658247296:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784858.696753:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784858.696771:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7302 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b161f expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.696847:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7309 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.696860:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7309 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.696870:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7309 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784858.696898:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x29:0x0]: returning size 0 00000004:00200000:0.0:1787784858.696902:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784858.696917:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7309 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.696940:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7302 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b161f expref: 62 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.696956:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7309 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b161f expref: 61 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.696964:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:0.0:1787784858.696968:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1787784858.696971:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b800, rc 0) 00000100:00100000:0.0:1787784858.697020:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+61:31512:x1874627658247296:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 288us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784858.697027:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1365 00000100:00100000:0.0:1787784858.699826:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.700185:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658247424 00000100:00100000:1.0:1787784858.700194:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658247424 00000100:00100000:1.0:1787784858.700239:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 381 00000100:00100000:1.0:1787784858.700249:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658247424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784858.700265:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784858.700351:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1626 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784858.700378:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784858.700396:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=649s v=(0 0) 00010000:00010000:1.0:1787784858.700400:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784858.700408:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.700423:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.700435:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640400, rc 0) 00000100:00100000:1.0:1787784858.700488:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658247424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 241us (670us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.700494:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 381 00000100:00100000:2.0:1787784858.703420:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.703504:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658247680 00000100:00100000:3.0:1787784858.703513:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658247680 00000100:00100000:3.0:1787784858.703589:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1366 00000100:00100000:3.0:1787784858.703598:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+61:31512:x1874627658247680:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784858.703703:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+61:31512:x1874627658247680:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 106us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784858.703711:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1366 00000100:00100000:2.0:1787784858.708004:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.708082:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658247808 00000100:00100000:1.0:1787784858.708091:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658247808 00000100:00100000:1.0:1787784858.708139:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 382 00000100:00100000:1.0:1787784858.708168:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:31512:x1874627658247808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784858.708184:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784858.708202:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b162d expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.708233:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.708242:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784858.708246:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784858.708251:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.708279:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.708292:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 12 pid: 10688 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.708306:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402187840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668437248:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784858.708381:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895036d10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784858.711002:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784858.711365:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402187840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668437248:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784858.711396:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.711415:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641c00, rc 0) 00000100:00100000:2.0:1787784858.711464:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:31512:x1874627658247808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3316us (3468us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.711472:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 382 00000100:00100000:3.0:1787784858.711946:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658247936 00000100:00100000:3.0:1787784858.711953:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658247936 00010000:00010000:3.0:1787784858.711980:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 12 pid: 10688 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784858.712005:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 410 00000100:00100000:3.0:1787784858.712012:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658247936:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784858.712024:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784858.712042:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 12 pid: 10688 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.712054:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 12 pid: 10688 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.712089:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007a4ce41d/0xb0ce63137acc7310 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1626 expref: 12 pid: 10688 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.712103:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784858.712108:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784858.712115:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 11 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.712126:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 11 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.712338:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 11 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784858.712352:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668437376:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784858.712400:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952482e0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784858.714726:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668437376:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784858.714748:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784858.714797:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658247936:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2785us (3801us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784858.714814:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 410 00000100:00100000:2.0:1787784858.718122:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.718218:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658248064 00000100:00100000:3.0:1787784858.718228:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658248064 00002000:00100000:0.0:1787784858.718305:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1d:0x0] (0->157) req@ffff989508853480 x1874627658248064/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:739/0 lens 648/0 e 0 to 0 dl 1787784869 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784858.718321:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989508853480 with x1874627658248064 ext(0->157) 00002000:00010000:0.0:1787784858.718326:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989508853480 00000100:00100000:0.0:1787784858.718354:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 118 00000100:00100000:0.0:1787784858.718363:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658248064:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1787784858.718815:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658248064:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 451us (701us total) trans 4294967380 rc 0/0 00000100:00100000:0.0:1787784858.718823:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 118 00002000:00100000:0.0:1787784858.718829:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1d:0x0] (0->157) req@ffff989508853480 x1874627658248064/t4294967380(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:739/0 lens 648/448 e 0 to 0 dl 1787784869 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784858.718836:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989508853480 with x1874627658248064 ext(0->157) 00002000:00010000:0.0:1787784858.718839:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989508853480 00000100:00100000:2.0:1787784858.725028:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.726290:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658248192 00000100:00100000:1.0:1787784858.726303:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658248192 00000100:00100000:1.0:1787784858.726358:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1367 00000100:00100000:1.0:1787784858.726367:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+61:31512:x1874627658248192:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1787784858.786566:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+61:31512:x1874627658248192:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 60189us (61536us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784858.786580:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1367 00000001:00080000:0.0:1787784858.786617:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968192 is committed 00000001:00080000:0.0:1787784858.786628:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968187 is committed 00000001:00080000:0.0:1787784858.786637:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968206 is committed 00000001:00080000:0.0:1787784858.786646:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968200 is committed 00000004:00080000:0.0:1787784858.786649:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 14 committed 00000020:00080000:0.0:1787784858.786736:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98954066d540 refcount 2 committed 0 result 0 batchid 31 00000020:00080000:0.0:1787784858.786741:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953efed200 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98954066d600 00000020:00080000:0.0:1787784858.786744:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953efeda00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9894035ed800 00000001:00080000:0.0:1787784858.786780:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968207 is committed 00000001:00080000:0.0:1787784858.786785:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968201 is committed 00000020:00080000:0.0:1787784858.786976:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 31 committed batchid 30 00000020:00080000:0.0:1787784858.787127:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30 00000100:00100000:0.0:1787784858.787144:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668437504:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1787784858.787216:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784858.787257:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668437504 00000100:00100000:0.0:1787784858.787264:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668437504 00000100:00100000:0.0:1787784858.787367:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:0.0:1787784858.787376:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668437504:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1787784858.787399:0:6670:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:26 count 1 00002000:00010000:0.0:1787784858.787427:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1a:0x0] initial LVB size: 158, mtime: 0x6a8f6e98, atime: 0x0, ctime: 0x6a8f6e98, blocks: 0xb 00010000:00010000:0.0:1787784858.787436:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.787451:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=649s v=(0 0) 00010000:00010000:0.0:1787784858.787456:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784858.787462:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.787476:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.787485:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.787494:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.787515:0:6670:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787784858.788179:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668437504:12345-0@lo:6:osp-syn-0-0.0 Request processed in 801us (960us total) trans 4294967370 rc 0/0 00000100:00100000:0.0:1787784858.788188:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00010000:00010000:0.0:1787784858.788216:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787784858.788222:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.788232:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150be00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784858.788236:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.788246:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.788261:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.788269:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0669d52/0xb0ce63137acc731e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.788356:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784858.788358:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784858.788359:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787784858.789757:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402ade580/2, rc 0, transno 4294967370 00000100:00100000:1.0:1787784858.789769:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668437504:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787784858.793115:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784858.795641:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658248320 00000100:00100000:2.0:1787784858.795654:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658248320 00000100:00100000:2.0:1787784858.795718:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 384 00000100:00100000:2.0:1787784858.795728:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658248320:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:3.0:1787784858.826940:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967380 is committed 00000100:00100000:1.0:1787784858.827342:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658248320:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 31548us (34167us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.827354:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 384 00000100:00100000:0.0:1787784858.835919:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.835979:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658248576 00000100:00100000:1.0:1787784858.835989:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658248576 00000100:00100000:1.0:1787784858.836046:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1368 00000100:00100000:1.0:1787784858.836057:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:31512:x1874627658248576:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784858.836118:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784858.836227:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7309 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b161f expref: 50 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.836255:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:1.0:1787784858.836261:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784858.836385:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.836400:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.836410:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784858.836683:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004f966cc3 reply state 00000000217f2d84 transno 4294968208 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.836699:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784858.836807:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:31512:x1874627658248576:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 752us (897us total) trans 4294968208 rc 0/0 00000100:00100000:1.0:1787784858.836815:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1368 00000100:00100000:3.0:1787784858.839848:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784858.839983:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.839996:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.840004:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.840011:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784858.840016:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.840024:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.840041:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:1.0:1787784858.840044:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784858.840048:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.840055:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7325 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.840222:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658248704 00000100:00100000:0.0:1787784858.840231:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658248704 00000100:00100000:0.0:1787784858.840281:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1369 00000100:00100000:0.0:1787784858.840291:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:31512:x1874627658248704:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784858.840312:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784858.840328:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1634 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.840381:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.840394:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.840412:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:0.0:1787784858.840415:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784858.840417:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150be00, rc 0) 00000100:00100000:0.0:1787784858.840471:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:31512:x1874627658248704:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 181us (630us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784858.840478:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1369 00000100:00100000:0.0:1787784858.847981:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784858.848083:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658248832 00000100:00100000:2.0:1787784858.848093:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658248832 00000100:00100000:2.0:1787784858.848222:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 385 00000100:00100000:2.0:1787784858.848233:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658248832:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1787784858.848479:0:6669:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:2.0:1787784858.848484:0:6669:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb mtime from disk: 0 -> 1787784856 00002000:00010000:2.0:1787784858.848487:0:6669:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb atime from disk: 0 -> 1787784856 00002000:00010000:2.0:1787784858.848489:0:6669:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb ctime from disk: 0 -> 1787784856 00002000:00010000:2.0:1787784858.848492:0:6669:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1787784858.848548:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658248832:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 316us (577us total) trans 4294967381 rc 0/0 00000100:00100000:2.0:1787784858.848556:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 385 00000100:00100000:2.0:1787784858.850556:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.850860:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658249088 00000100:00100000:0.0:1787784858.850867:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658249088 00000100:00100000:0.0:1787784858.850904:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1370 00000100:00100000:0.0:1787784858.850911:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:31512:x1874627658249088:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1787784858.850963:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.850976:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.850982:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.851007:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.851016:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 51 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.851027:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668437632:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784858.851121:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950a630b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784858.854258:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.854772:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668437632:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784858.854799:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.855212:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658249216 00000100:00100000:1.0:1787784858.855238:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658249216 00010000:00010000:1.0:1787784858.855314:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.855484:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 411 00000100:00100000:1.0:1787784858.855533:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:2356:x1874627658249216:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784858.855572:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784858.855580:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.855593:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.855613:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc72d8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b160a expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.855646:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:1.0:1787784858.855669:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.855679:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.855706:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.855729:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784858.855821:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2356:x1874627658249216:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 328us (1571us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.855829:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 411 00010000:00010000:0.0:1787784858.858437:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858452:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858481:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858486:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858491:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858499:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784858.858502:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858507:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858522:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:0.0:1787784858.858526:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787784858.858530:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.858534:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7333 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.858611:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:31512:x1874627658249088:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 7694us (8059us total) trans 0 rc -22/-22 00000100:00100000:0.0:1787784858.858621:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1370 00000100:00100000:2.0:1787784858.866499:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.867210:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658249344 00000100:00100000:0.0:1787784858.867220:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658249344 00000100:00100000:0.0:1787784858.867286:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:0.0:1787784858.867297:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658249344:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784858.867339:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.867360:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.867375:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784858.867379:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784858.867384:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.867416:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.867430:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 12 pid: 6669 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.867444:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668437760:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784858.867520:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950a630b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784858.870045:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668437760:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784858.870085:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784858.873592:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784858.874225:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658249472 00000100:00100000:0.0:1787784858.874236:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658249472 00010000:00010000:0.0:1787784858.874372:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 13 pid: 6669 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784858.874682:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 412 00000100:00100000:0.0:1787784858.874693:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c94450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658249472:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784858.874713:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784858.874733:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 13 pid: 6669 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.874747:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 13 pid: 6669 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.874769:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7317 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b162d expref: 13 pid: 6669 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.874785:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784858.874790:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784858.874795:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.874805:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.874840:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784858.874896:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c94450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658249472:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 205us (1314us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784858.874904:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 412 00010000:00010000:2.0:1787784858.875507:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875532:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875575:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875602:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875611:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875619:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784858.875623:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875646:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875666:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784858.875690:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000acbc6c6b/0xb0ce63137acc733a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784858.875819:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658249344:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 8523us (9344us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.875827:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 386 00000100:00100000:2.0:1787784858.879894:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.882615:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658249728 00000100:00100000:1.0:1787784858.882631:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658249728 00000100:00100000:1.0:1787784858.882709:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1371 00000100:00100000:1.0:1787784858.882723:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:31512:x1874627658249728:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784858.882815:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.882834:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.882849:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.883094:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.883106:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.883114:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.883169:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.883182:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 51 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784858.883196:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668437888:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784858.883291:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504c5b000 going to sleep for 11 seconds 00000100:00100000:1.0:1787784858.891398:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668437888:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784858.891427:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784858.900570:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.900649:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658249856 00000100:00100000:3.0:1787784858.900667:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658249856 00010000:00010000:3.0:1787784858.900706:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784858.900744:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 413 00000100:00100000:3.0:1787784858.900753:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:2357:x1874627658249856:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784858.900766:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784858.900773:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.900785:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.900806:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc732c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1634 expref: 52 pid: 5813 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.900822:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:3.0:1787784858.900828:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.900837:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.900844:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.900879:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784858.900935:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:2357:x1874627658249856:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 183us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784858.900943:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 413 00010000:00010000:0.0:1787784858.901116:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784858.901124:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784858.901348:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968209 00000001:00080000:0.0:1787784858.901491:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968210 00000001:00080000:0.0:1787784858.901545:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968211 00000100:00100000:3.0:1787784858.902266:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668438016:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787784858.902318:0:20438:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784858.902351:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066436c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784858.902783:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668438016 00000100:00100000:0.0:1787784858.902790:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668438016 00000100:00100000:0.0:1787784858.902850:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:0.0:1787784858.902859:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:20438:x1874627668438016:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787784858.902936:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:20438:x1874627668438016:12345-0@lo:1000:mdt00_004.0 Request processed in 78us (617us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784858.902943:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00000100:00100000:0.0:1787784858.902986:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668438016:0@lo:1000:mdt00_004.0 00000001:00080000:0.0:1787784858.903390:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968212 00000100:00100000:0.0:1787784858.903451:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668438144:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787784858.903483:0:20438:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784858.903502:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950a630780 going to sleep for 11 seconds 00000100:00100000:0.0:1787784858.903516:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668438144 00000100:00100000:0.0:1787784858.903519:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668438144 00000100:00100000:0.0:1787784858.903541:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 68 00000100:00100000:0.0:1787784858.903546:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:20438:x1874627668438144:12345-0@lo:1000:mdt00_004.0 00000100:00080000:3.0:1787784858.903725:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784858.904467:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:20438:x1874627668438144:12345-0@lo:1000:mdt00_004.0 Request processed in 918us (982us total) trans 4294967382 rc 0/0 00000100:00100000:0.0:1787784858.904477:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 68 00000100:00100000:3.0:1787784858.904610:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668438144:0@lo:1000:mdt00_004.0 00000001:00080000:3.0:1787784858.904868:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968213 00010000:00010000:3.0:1787784858.904903:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.904913:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.904920:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.904926:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784858.904930:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.904939:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.904953:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff989520131c00) 00010000:00010000:3.0:1787784858.904957:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784858.904962:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.904967:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7348 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.904996:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905002:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905008:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905013:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784858.905016:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905022:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905029:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520131400) 00010000:00010000:3.0:1787784858.905031:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784858.905037:0:20438:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15f5 expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905047:0:20438:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15f5 expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905056:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15f5 expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.905259:0:20438:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a000/0xb0ce63137acc727d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b15f5 expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784858.905275:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668438272:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784858.905388:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066436c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784858.908523:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.908600:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658249984 00000100:00100000:1.0:1787784858.908607:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658249984 00000100:00100000:1.0:1787784858.908684:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 276 00000100:00100000:1.0:1787784858.908694:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d1450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:31510:x1874627658249984:12345-192.168.203.43@tcp:49:lseek_test.0 00000100:00100000:1.0:1787784858.908832:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d1450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:31510:x1874627658249984:12345-192.168.203.43@tcp:49:lseek_test.0 Request processed in 139us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.908841:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 276 00000100:00100000:3.0:1787784858.910326:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668438272:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784858.910356:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.910368:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7276 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.910408:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.910422:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.910434:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.910445:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911667:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911695:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911700:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784858.911704:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911712:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911725:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520131400) 00010000:00010000:3.0:1787784858.911728:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784858.911732:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911737:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc734f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911752:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911758:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911763:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911768:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784858.911770:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911775:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911781:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff989520131400) 00010000:00010000:3.0:1787784858.911783:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784858.911786:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.911791:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7341 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787784858.912172:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784858.912178:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784858.912663:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:1.0:1787784858.912672:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784858.913397:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+55:31512:x1874627658249728:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 30670us (33520us total) trans 4294968214 rc 0/0 00000100:00100000:3.0:1787784858.913409:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1371 00000020:00080000:3.0:1787784858.913415:0:20438:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:2.0:1787784858.913596:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.917772:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658250112 00000100:00100000:1.0:1787784858.917785:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658250112 00000100:00100000:1.0:1787784858.917952:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 387 00000100:00100000:1.0:1787784858.917965:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+14:31510:x1874627658250112:12345-192.168.203.43@tcp:101:lseek_test.0 00010000:00010000:1.0:1787784858.917983:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784858.918004:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7356 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b163b expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784858.918030:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1d:0x0] initial LVB size: 158, mtime: 0x6a8f6e98, atime: 0x6a8f6e98, ctime: 0x6a8f6e98, blocks: 0xb 00010000:00010000:1.0:1787784858.918052:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=650s v=(0 0) 00010000:00010000:1.0:1787784858.918056:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784858.918064:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7356 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b163b expref: 15 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.918101:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003743046c/0xb0ce63137acc7356 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b163b expref: 15 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.918114:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641c00, rc 0) 00000100:00100000:1.0:1787784858.918209:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:31510:x1874627658250112:12345-192.168.203.43@tcp:101:lseek_test.0 Request processed in 247us (4614us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784858.918217:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 387 00000100:00100000:0.0:1787784858.921554:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.921649:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658250368 00000100:00100000:1.0:1787784858.921659:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658250368 00000100:00100000:1.0:1787784858.921719:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 277 00000100:00100000:1.0:1787784858.921730:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+55:31512:x1874627658250368:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1787784858.922429:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x29:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1787784858.922538:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968215 00010000:00010000:1.0:1787784858.922590:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.922608:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.922623:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.922631:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.922638:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.922649:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784858.922672:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787784858.922764:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+57:31512:x1874627658250368:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1034us (1218us total) trans 4294968216 rc 0/0 00000100:00100000:1.0:1787784858.922773:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 277 00000020:00080000:1.0:1787784858.922779:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1787784858.922899:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784858.922906:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.922914:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784858.922919:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.922927:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.922940:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff98951c407500) 00010000:00010000:3.0:1787784858.922944:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784858.922948:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 1/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.922953:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc735d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784858.922968:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784858.922971:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784858.922972:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784858.923305:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784858.923379:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658250496 00000100:00100000:3.0:1787784858.923386:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658250496 00000100:00100000:3.0:1787784858.923419:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 388 00000100:00100000:3.0:1787784858.923427:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658250496:12345-192.168.203.43@tcp:23:lseek_test.0 00000001:00080000:0.0:1787784858.955912:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967382 is committed 00000001:00080000:0.0:1787784858.955932:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967381 is committed 00000100:00100000:2.0:1787784858.966820:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2357:x1874627658250496:12345-192.168.203.43@tcp:23:lseek_test.0 Request processed in 43373us (43496us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784858.966834:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 388 00000100:00100000:3.0:1787784858.982320:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784858.982421:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658250752 00000100:00100000:1.0:1787784858.982430:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658250752 00000100:00100000:1.0:1787784858.982604:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 278 00000100:00100000:1.0:1787784858.982615:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:31510:x1874627658250752:12345-192.168.203.43@tcp:35:lseek_test.0 00000100:00100000:1.0:1787784858.982846:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:31510:x1874627658250752:12345-192.168.203.43@tcp:35:lseek_test.0 Request processed in 233us (534us total) trans 4294968217 rc 0/0 00000100:00100000:1.0:1787784858.982855:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 278 00000100:00100000:1.0:1787784859.123258:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787784859.123290:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895407383c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668438400:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784859.123385:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784859.124085:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668438400 00000100:00100000:0.0:1787784859.124098:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668438400 00000100:00100000:0.0:1787784859.124186:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:0.0:1787784859.124308:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668438400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784859.124357:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784859.124426:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668438400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 124us (1042us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784859.124435:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1787784859.124710:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895407383c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668438400:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787784859.569543:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784859.962485:0:21878:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784859.962501:0:21878:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784859.962560:0:21878:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 41 count 4 00000040:00080000:2.0:1787784859.962567:0:21878:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784860.596512:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784861.169973:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784861.170004:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540739a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668438528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784861.170116:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784861.170230:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668438528 00000100:00100000:0.0:1787784861.170239:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668438528 00000100:00080000:3.0:1787784861.170350:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784861.170371:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1787784861.170383:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668438528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784861.170438:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784861.170496:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668438528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 115us (380us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784861.170504:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1787784861.170541:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668438784:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.170581:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784861.170607:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784861.170613:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668438912:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.170637:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784861.170675:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668438784 00000100:00100000:3.0:1787784861.170680:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668438784 00000100:00100000:3.0:1787784861.170711:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668438912 00000100:00100000:3.0:1787784861.170714:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668438912 00000100:00100000:3.0:1787784861.170742:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1787784861.170749:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668438784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.170789:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668438784:12345-0@lo:400:kworker.0 Request processed in 41us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784861.170797:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1787784861.170810:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1787784861.170816:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668438912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.170846:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668438912:12345-0@lo:400:kworker.0 Request processed in 30us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784861.170852:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00080000:3.0:1787784861.170874:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784861.170906:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506337480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668438912:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.170919:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668438784:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.171646:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668438656:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.171691:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784861.171723:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668439168:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.171746:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784861.171771:0:3279:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967382 gen 2 00000004:00080000:3.0:1787784861.171777:0:3279:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967382 committed transno 4294967382 00000100:00100000:3.0:1787784861.172120:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540739a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668438528:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784861.172137:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784861.172144:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668439040:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.172196:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784861.172197:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439168 00000100:00100000:0.0:1787784861.172205:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439168 00000100:00100000:3.0:1787784861.172242:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668438656 00000100:00100000:3.0:1787784861.172248:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668438656 00000100:00100000:0.0:1787784861.172259:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:0.0:1787784861.172271:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668439168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.172288:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:3.0:1787784861.172298:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668438656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784861.172327:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668439168:12345-0@lo:400:kworker.0 Request processed in 58us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784861.172344:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668438656:12345-0@lo:400:kworker.0 Request processed in 48us (651us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784861.172351:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00100000:3.0:1787784861.172386:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439040 00000100:00100000:3.0:1787784861.172390:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439040 00000100:00100000:3.0:1787784861.172420:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1787784861.172428:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3279:x1874627668439040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.172458:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3279:x1874627668439040:12345-0@lo:400:kworker.0 Request processed in 32us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784861.172464:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1787784861.172494:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668439040:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.172503:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668438656:0@lo:400:kworker.0 00000100:00100000:3.0:1787784861.172512:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668439168:0@lo:400:kworker.0 00000100:00100000:0.0:1787784861.172902:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00080000:0.0:1787784861.172933:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784861.172937:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784861.618682:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784862.002510:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784862.002524:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784862.003009:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 41 count 3 00000040:00080000:1.0:1787784862.003018:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784862.472815:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784862.472863:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784862.472910:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784862.472933:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784862.472957:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784862.472978:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784862.473002:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784862.473036:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251136 00000100:00100000:0.0:1787784862.473058:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251136 00000100:00100000:0.0:1787784862.473128:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251264 00000100:00100000:0.0:1787784862.473132:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251264 00000100:00100000:0.0:1787784862.473171:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251520 00000100:00100000:0.0:1787784862.473173:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251520 00000100:00100000:0.0:1787784862.473191:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251648 00000100:00100000:0.0:1787784862.473193:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251648 00000100:00100000:0.0:1787784862.473210:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:0.0:1787784862.473220:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658251136:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784862.473310:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658251136:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 91us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784862.473319:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:0.0:1787784862.473504:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 431 00000100:00100000:0.0:1787784862.473511:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658251264:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784862.473536:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658251264:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (603us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784862.473542:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 431 00000100:00100000:0.0:1787784862.473553:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 432 00000100:00100000:0.0:1787784862.473558:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658251520:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784862.473576:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658251520:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 18us (620us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784862.473582:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 432 00000100:00100000:0.0:1787784862.473591:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 433 00000100:00100000:0.0:1787784862.473597:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658251648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784862.473617:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658251648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (614us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784862.473622:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 433 00000100:00100000:0.0:1787784862.473646:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251008 00000100:00100000:0.0:1787784862.473649:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251008 00000100:00100000:0.0:1787784862.473680:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251392 00000100:00100000:0.0:1787784862.473683:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251392 00000100:00100000:0.0:1787784862.473704:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:0.0:1787784862.473710:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:2355:x1874627658251008:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784862.473746:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:2355:x1874627658251008:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 36us (883us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784862.473753:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:0.0:1787784862.473766:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 467 00000100:00100000:0.0:1787784862.473772:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:2355:x1874627658251392:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784862.473800:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:2355:x1874627658251392:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784862.473806:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 467 00000100:00080000:0.0:1787784862.473824:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784862.473829:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784862.473831:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784862.474863:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658250880 00000100:00100000:3.0:1787784862.474878:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658250880 00000100:00100000:3.0:1787784862.474943:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 603 00000100:00100000:3.0:1787784862.474953:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658250880:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784862.475011:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658250880:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 59us (2212us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784862.475018:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 603 00010000:00010000:0.0:1787784862.641443:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784863.356081:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967370 is committed 00010000:00010000:0.0:1787784863.668328:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784863.800220:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968213 is committed 00000020:00080000:0.0:1787784863.800245:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989504d9c840 refcount 2 committed 0 result 0 batchid 32 00000020:00080000:0.0:1787784863.800252:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895032bb500 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9895113b50c0 00000020:00080000:0.0:1787784863.800258:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9895032bb600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989518f0dc00 00000020:00080000:0.0:1787784863.800293:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 31 batchid updated 00000001:00080000:0.0:1787784863.800307:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968217 is committed 00000001:00080000:0.0:1787784863.800313:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968216 is committed 00000020:00080000:3.0:1787784863.810927:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 32 committed batchid 31 00000020:00080000:3.0:1787784863.812724:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 31 00000100:00100000:1.0:1787784864.241434:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954073ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668439296:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784864.241539:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784864.241722:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439296 00000100:00100000:2.0:1787784864.241733:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439296 00000100:00100000:2.0:1787784864.241831:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:2.0:1787784864.241849:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668439296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784864.241895:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3857711104 avail 3855613952 00000100:00080000:3.0:1787784864.241992:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784864.248994:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668439296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 7137us (7439us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784864.249015:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1787784864.251463:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967370 gen 2 00000004:00080000:1.0:1787784864.251481:0:3276:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967369 committed transno 4294967370 00000004:00080000:1.0:1787784864.251516:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402ade580, transno 4294967370 00000100:00100000:1.0:1787784864.251638:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954073ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668439296:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787784864.691444:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784865.333262:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784865.333295:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251776 00000100:00100000:3.0:1787784865.333303:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251776 00000100:00100000:3.0:1787784865.333361:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1372 00000100:00100000:3.0:1787784865.333371:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31491:x1874627658251776:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1787784865.333455:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:31491:x1874627658251776:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 84us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784865.333462:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1372 00000100:00080000:3.0:1787784865.333484:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784865.713559:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784866.289370:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668439424:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784866.289444:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784866.289539:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439424 00000100:00100000:3.0:1787784866.289547:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439424 00000100:00100000:3.0:1787784866.289615:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:3.0:1787784866.289628:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668439424:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784866.289663:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784866.289709:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668439424:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784866.289716:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00080000:3.0:1787784866.289735:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784866.289772:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668439424:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784866.289999:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668439552:0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.290048:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784866.290076:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784866.290083:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668439680:0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.290112:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784866.290141:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439680 00000100:00100000:0.0:1787784866.290166:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439680 00000100:00100000:0.0:1787784866.290199:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:0.0:1787784866.290206:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668439680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.290244:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668439680:12345-0@lo:400:kworker.0 Request processed in 39us (131us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784866.290250:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:0.0:1787784866.290286:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668439680:0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.290322:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439552 00000100:00100000:0.0:1787784866.290326:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439552 00000100:00100000:0.0:1787784866.290351:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 604 00000100:00100000:0.0:1787784866.290357:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668439552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.290392:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668439552:12345-0@lo:400:kworker.0 Request processed in 36us (344us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784866.290400:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 604 00000100:00100000:1.0:1787784866.290434:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668439552:0@lo:400:kworker.0 00000100:00100000:3.0:1787784866.292832:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784866.292853:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668440064:0@lo:400:kworker.0 00000100:00100000:3.0:1787784866.292911:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784866.293673:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ee580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668439936:0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.293716:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784866.293746:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784866.293753:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668439808:0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.293780:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784866.293813:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439936 00000100:00100000:0.0:1787784866.293820:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439936 00000100:00100000:0.0:1787784866.293884:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1787784866.293893:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668439936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.293951:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668439936:12345-0@lo:400:kworker.0 Request processed in 59us (235us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784866.293958:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1787784866.293989:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668440064 00000100:00100000:0.0:1787784866.293994:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668440064 00000100:00100000:1.0:1787784866.294025:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668439808 00000100:00100000:1.0:1787784866.294031:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668439808 00000100:00100000:0.0:1787784866.294038:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:0.0:1787784866.294047:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668440064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784866.294075:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:1.0:1787784866.294086:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668439808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.294093:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668440064:12345-0@lo:400:kworker.0 Request processed in 45us (1179us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784866.294100:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:1.0:1787784866.294144:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668439808:12345-0@lo:400:kworker.0 Request processed in 60us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784866.294174:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00080000:1.0:1787784866.294214:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784866.294218:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784866.294534:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668439808:0@lo:400:kworker.0 00000100:00100000:1.0:1787784866.294550:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ee580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668439936:0@lo:400:kworker.0 00000100:00100000:0.0:1787784866.294562:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668440064:0@lo:400:kworker.0 00010000:00010000:2.0:1787784866.744759:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787784867.094326:0:22075:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12s: race between restore requests ==== 18:54:25 (1787784865) 00000100:00100000:0.0:1787784867.439238:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.439407:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658251904 00000100:00100000:2.0:1787784867.439421:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658251904 00000100:00100000:2.0:1787784867.439512:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1373 00000100:00100000:2.0:1787784867.439526:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:32127:x1874627658251904:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00080000:1.0:1787784867.439785:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784867.439902:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:32127:x1874627658251904:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 378us (682us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784867.439912:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1373 00000100:00100000:0.0:1787784867.451683:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784867.451979:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658252032 00000100:00100000:3.0:1787784867.451992:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658252032 00000100:00100000:3.0:1787784867.452106:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1374 00000100:00100000:3.0:1787784867.452117:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:32128:x1874627658252032:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784867.452185:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784867.452216:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7364 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1642 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452305:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452323:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452339:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452356:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452364:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452371:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452705:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452715:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452722:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452728:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784867.452733:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452741:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.452755:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:3.0:1787784867.452758:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.452762:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784867.452963:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784867.453435:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc736b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453457:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453463:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453469:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453474:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784867.453477:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453484:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453495:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff9895198b5a00) 00010000:00010000:3.0:1787784867.453498:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.453502:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453508:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc7372 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453933:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7364 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1642 expref: 103 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453949:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:3.0:1787784867.453952:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.453957:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7364 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1642 expref: 103 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.453970:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784867.454974:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:32128:x1874627658252032:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2850us (3298us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784867.454987:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1374 00000100:00100000:0.0:1787784867.473262:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.474466:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658252288 00000100:00100000:2.0:1787784867.474483:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658252288 00000100:00100000:2.0:1787784867.474566:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1375 00000100:00100000:2.0:1787784867.474580:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:32128:x1874627658252288:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:2.0:1787784867.474674:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784867.474711:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc70cb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b155b expref: 102 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.474734:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784867.474741:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784867.475098:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.475117:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.475126:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.475203:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 47 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.475222:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 47 pid: 10850 timeout: 758 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784867.475238:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668440192:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784867.475364:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d540 going to sleep for 11 seconds 00000100:00100000:1.0:1787784867.484625:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784867.487027:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658252416 00000100:00100000:3.0:1787784867.487049:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658252416 00010000:00010000:3.0:1787784867.487122:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 48 pid: 10850 timeout: 758 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784867.487222:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 414 00000100:00100000:3.0:1787784867.487235:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2356:x1874627658252416:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784867.487255:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784867.487263:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 48 pid: 10850 timeout: 758 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.487280:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 48 pid: 10850 timeout: 758 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.487312:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:3.0:1787784867.487319:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.487329:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.487337:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.487363:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784867.487446:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2356:x1874627658252416:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 214us (2835us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784867.487455:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 414 00010000:00010000:2.0:1787784867.489265:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc710a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1570 expref: 47 pid: 10850 timeout: 758 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784867.489295:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668440192:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784867.489325:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.489337:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.489344:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.489375:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.489388:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.489398:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784867.495496:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 512 00010000:00010000:2.0:1787784867.495525:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.496958:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.496969:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.496976:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.496982:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.496991:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.497083:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784867.497087:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.497094:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.497100:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc7379 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784867.497139:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000021425471 reply state 0000000033847fda transno 4294968218 ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784867.497175:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784867.497478:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:32128:x1874627658252288:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 22895us (24223us total) trans 4294968218 rc 0/0 00000100:00100000:2.0:1787784867.497489:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1375 00010000:00010000:0.0:1787784867.513245:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.513269:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.513283:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.513291:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784867.513297:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.513306:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.513330:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff989520131500) 00010000:00010000:0.0:1787784867.513334:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784867.513341:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.513347:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc7380 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784867.517837:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784867.518252:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658252544 00000100:00100000:3.0:1787784867.518266:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658252544 00000100:00100000:3.0:1787784867.518339:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1376 00000100:00100000:3.0:1787784867.518351:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+102:32129:x1874627658252544:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784867.518384:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784867.518418:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7387 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1649 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.518509:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc738e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.518525:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc738e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.518538:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc738e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784867.518617:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:3.0:1787784867.518621:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 33 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784867.518666:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc738e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.518696:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7387 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1649 expref: 104 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.518725:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc738e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1649 expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.518735:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:3.0:1787784867.518738:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784867.518741:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784867.518827:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:32129:x1874627658252544:12345-192.168.203.43@tcp:101:bash.0 Request processed in 477us (1005us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784867.518835:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1376 00000100:00100000:1.0:1787784867.524985:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784867.525189:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658252672 00000100:00100000:3.0:1787784867.525202:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658252672 00000100:00100000:3.0:1787784867.525272:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1377 00000100:00100000:3.0:1787784867.525283:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:32129:x1874627658252672:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784867.525311:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784867.525338:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7395 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1650 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.525429:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.525448:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.525460:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.525475:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.525484:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.525491:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536715:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc73aa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536758:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc73aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536782:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc73aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784867.536875:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 512 00000004:00200000:2.0:1787784867.536880:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784867.536911:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc73aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536929:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536937:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536945:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536952:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.536957:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536966:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.536981:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:2.0:1787784867.536984:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.536990:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537001:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc739c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537036:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537043:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537050:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537055:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.537058:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537064:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537072:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff9895198b5000) 00010000:00010000:2.0:1787784867.537075:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.537078:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537084:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc73a3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537137:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7395 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1650 expref: 105 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537181:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2000/0xb0ce63137acc73aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1650 expref: 104 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.537191:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784867.537194:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784867.537197:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2000, rc 0) 00000100:00100000:2.0:1787784867.537304:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32129:x1874627658252672:12345-192.168.203.43@tcp:101:bash.0 Request processed in 12021us (12332us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784867.537315:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1377 00000100:00100000:0.0:1787784867.541273:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.541392:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658252800 00000100:00100000:2.0:1787784867.541405:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658252800 00000100:00100000:2.0:1787784867.541482:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1378 00000100:00100000:2.0:1787784867.541496:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32129:x1874627658252800:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:2.0:1787784867.541525:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784867.541563:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc73b1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1657 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.541660:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.541679:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.541694:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.541713:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.541721:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.541729:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542000:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542022:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542030:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542037:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895013a2400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.542042:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542051:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542067:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784867.542070:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.542076:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542083:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2400/0xb0ce63137acc73b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542098:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542104:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542111:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542116:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.542119:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542125:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542132:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff989520131100) 00010000:00010000:2.0:1787784867.542135:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.542138:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542144:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542216:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc73b1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1657 expref: 105 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542228:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784867.542231:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.542235:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc73b1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1657 expref: 105 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.542245:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895013a2a00, rc 0) 00000100:00100000:2.0:1787784867.542317:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32129:x1874627658252800:12345-192.168.203.43@tcp:101:bash.0 Request processed in 824us (1056us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784867.542330:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1378 00000100:00100000:1.0:1787784867.562290:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.563300:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658252928 00000100:00100000:2.0:1787784867.563318:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658252928 00000100:00100000:2.0:1787784867.563410:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1379 00000100:00100000:2.0:1787784867.563422:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+47:32130:x1874627658252928:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784867.563450:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784867.563477:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73c6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b165e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.563877:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.563896:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.563910:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784867.563985:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784867.563989:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 33 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784867.565235:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.565298:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73c6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 49 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.565325:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.565341:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:2.0:1787784867.565344:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784867.565349:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351800, rc 0) 00000100:00100000:2.0:1787784867.565516:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+48:32130:x1874627658252928:12345-192.168.203.43@tcp:101:rm.0 Request processed in 2094us (3238us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784867.565527:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1379 00000100:00100000:1.0:1787784867.576677:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.578863:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253056 00000100:00100000:2.0:1787784867.578884:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253056 00000100:00100000:2.0:1787784867.578995:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1380 00000100:00100000:2.0:1787784867.579018:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+48:32130:x1874627658253056:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784867.579047:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784867.579079:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73d4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1665 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579220:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579240:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579253:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579270:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579277:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579287:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579431:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc73e9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579440:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc73e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579448:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc73e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784867.579486:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 512 00000004:00200000:2.0:1787784867.579491:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784867.579518:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc73e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579534:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579541:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579548:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579553:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.579558:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579565:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579579:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:2.0:1787784867.579582:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.579588:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579595:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc73db lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579611:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579617:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579624:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579629:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.579631:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579638:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579657:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff989520131100) 00010000:00010000:2.0:1787784867.579659:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.579662:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579668:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc73e2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579708:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc73d4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1665 expref: 50 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579723:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350c00/0xb0ce63137acc73e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1665 expref: 49 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.579731:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784867.579734:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1787784867.579737:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403350c00, rc 0) 00000100:00100000:2.0:1787784867.579824:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32130:x1874627658253056:12345-192.168.203.43@tcp:101:rm.0 Request processed in 818us (3160us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784867.579832:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1380 00000100:00100000:0.0:1787784867.585720:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784867.586091:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253184 00000100:00100000:3.0:1787784867.586104:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253184 00000100:00100000:3.0:1787784867.586192:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1381 00000100:00100000:3.0:1787784867.586203:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32130:x1874627658253184:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784867.586230:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784867.586257:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc73f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b166c expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586339:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586357:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586370:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586385:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586393:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586399:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586480:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586494:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586501:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586507:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784867.586512:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586521:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586536:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:3.0:1787784867.586539:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.586544:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586550:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc73f7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586564:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586570:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586575:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586580:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784867.586583:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586588:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586595:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff9895198b5e00) 00010000:00010000:3.0:1787784867.586597:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.586599:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586605:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc73fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586645:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc73f0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b166c expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586663:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:3.0:1787784867.586665:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.586669:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc73f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b166c expref: 50 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.586680:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784867.586917:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32130:x1874627658253184:12345-192.168.203.43@tcp:101:rm.0 Request processed in 713us (1210us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784867.586926:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1381 00000100:00100000:0.0:1787784867.592304:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784867.592418:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784867.592454:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784867.592484:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784867.592516:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784867.592554:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253568 00000100:00100000:0.0:1787784867.592568:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253568 00000100:00100000:0.0:1787784867.592652:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253824 00000100:00100000:0.0:1787784867.592657:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253824 00000100:00100000:0.0:1787784867.592673:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253440 00000100:00100000:0.0:1787784867.592675:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253440 00000100:00100000:0.0:1787784867.592689:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253696 00000100:00100000:0.0:1787784867.592691:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253696 00000100:00100000:0.0:1787784867.592707:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 435 00000100:00100000:0.0:1787784867.592718:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658253568:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784867.593106:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00100000:3.0:1787784867.593118:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658253824:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784867.593231:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658253824:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 115us (776us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784867.593239:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00100000:3.0:1787784867.593268:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 437 00000100:00100000:3.0:1787784867.593273:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658253440:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784867.593296:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658253440:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (812us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784867.593302:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 437 00000100:00100000:3.0:1787784867.593313:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 438 00000100:00100000:3.0:1787784867.593318:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658253696:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784867.593337:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658253696:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 18us (820us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784867.593342:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 438 00000100:00100000:0.0:1787784867.597994:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2356:x1874627658253568:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 5259us (5558us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784867.598023:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 435 00000100:00080000:0.0:1787784867.598100:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784867.598106:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784867.598260:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.598748:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253952 00000100:00100000:2.0:1787784867.598760:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253952 00000100:00100000:2.0:1787784867.598826:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1382 00000100:00100000:2.0:1787784867.598837:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32130:x1874627658253952:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784867.598856:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784867.598884:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7405 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1673 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.598974:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.598991:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599005:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599033:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599040:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599046:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599136:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599144:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599187:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599194:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.599198:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599207:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599222:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784867.599226:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.599231:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599237:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc740c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599253:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599260:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599265:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599270:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.599273:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599279:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599286:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff989520131100) 00010000:00010000:2.0:1787784867.599288:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.599291:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599297:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7413 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599337:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7405 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1673 expref: 50 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599348:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784867.599350:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.599354:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7405 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1673 expref: 50 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.599362:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:2.0:1787784867.599424:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32130:x1874627658253952:12345-192.168.203.43@tcp:101:rm.0 Request processed in 589us (1165us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784867.599433:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1382 00000100:00100000:1.0:1787784867.608766:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658253312 00000100:00100000:1.0:1787784867.608789:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658253312 00000100:00100000:1.0:1787784867.608888:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 605 00000100:00100000:1.0:1787784867.608902:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658253312:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784867.609025:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658253312:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 126us (16738us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784867.609033:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 605 00000100:00100000:2.0:1787784867.629717:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784867.629842:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658254080 00000100:00100000:1.0:1787784867.629857:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658254080 00000100:00100000:1.0:1787784867.629942:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1383 00000100:00100000:1.0:1787784867.629955:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32131:x1874627658254080:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784867.629981:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784867.630017:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc741a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b167a expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630107:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630125:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630139:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630205:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630212:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630219:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630307:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630316:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630323:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630330:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784867.630335:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630343:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630358:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:1.0:1787784867.630361:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784867.630366:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630372:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7421 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630390:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630396:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630401:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630406:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784867.630410:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630416:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630422:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff98951c407300) 00010000:00010000:1.0:1787784867.630425:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784867.630428:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630434:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7428 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630472:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc741a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b167a expref: 50 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630484:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:1.0:1787784867.630486:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784867.630490:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc741a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b167a expref: 50 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.630500:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784867.630582:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32131:x1874627658254080:12345-192.168.203.43@tcp:101:cp.0 Request processed in 629us (882us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784867.630590:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1383 00000100:00100000:2.0:1787784867.635624:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784867.639421:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658254208 00000100:00100000:0.0:1787784867.639442:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658254208 00000100:00100000:0.0:1787784867.639548:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1384 00000100:00100000:0.0:1787784867.639562:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32131:x1874627658254208:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784867.639593:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784867.639623:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc742f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1681 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639732:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639752:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639767:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639786:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639794:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639800:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639894:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639902:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639910:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639918:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784867.639923:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639931:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639946:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:0.0:1787784867.639949:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784867.639955:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639961:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7436 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639978:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639985:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639991:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.639996:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150be00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784867.639999:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.640005:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.640023:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff989520108f00) 00010000:00010000:0.0:1787784867.640025:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784867.640028:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.640034:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc743d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.640079:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc742f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1681 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.640091:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:0.0:1787784867.640094:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784867.640098:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc742f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1681 expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784867.640114:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a200, rc 0) 00000100:00100000:0.0:1787784867.640473:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32131:x1874627658254208:12345-192.168.203.43@tcp:101:cp.0 Request processed in 912us (4862us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784867.640483:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1384 00000100:00100000:0.0:1787784867.647119:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.650263:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658254464 00000100:00100000:2.0:1787784867.650284:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658254464 00000100:00100000:2.0:1787784867.650384:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1385 00000100:00100000:2.0:1787784867.650397:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32131:x1874627658254464:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784867.650436:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784867.650467:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc7444 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1688 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.650677:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.650698:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.650711:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.650730:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.650737:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.650745:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784867.651937:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1b:0x0], index 0: 0 00010000:00010000:2.0:1787784867.658274:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658298:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658311:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658668:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658675:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658681:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658686:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784867.658690:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658697:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658709:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784867.658712:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784867.658717:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658723:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc744b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784867.658737:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008555a83a reply state 000000004b91f034 transno 4294968219 ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784867.658745:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784867.658788:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895013a2a00/0xb0ce63137acc7444 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 52 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658802:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.658810:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784867.658813:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784867.658816:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351400, rc 0) 00000100:00100000:2.0:1787784867.658952:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32131:x1874627658254464:12345-192.168.203.43@tcp:101:cp.0 Request processed in 8556us (11852us total) trans 4294968219 rc 0/0 00000100:00100000:2.0:1787784867.658961:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1385 00000100:00100000:2.0:1787784867.666707:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784867.666714:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.666734:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.666745:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.666752:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784867.666758:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.666766:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.666790:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff989520131100) 00010000:00010000:1.0:1787784867.666795:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784867.666801:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.666808:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7452 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784867.666824:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658254592 00000100:00100000:3.0:1787784867.666834:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658254592 00000100:00100000:3.0:1787784867.666892:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 389 00000100:00100000:3.0:1787784867.666902:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658254592:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784867.666919:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784867.666938:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b168f expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784867.666962:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1787784867.666981:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=658s v=(0 0) 00010000:00010000:3.0:1787784867.666986:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784867.666996:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.668736:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.668752:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784867.668813:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658254592:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1913us (2118us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784867.668822:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 389 00000100:00100000:0.0:1787784867.676860:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.676958:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658254720 00000100:00100000:2.0:1787784867.676968:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658254720 00000100:00100000:2.0:1787784867.677039:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 390 00000100:00100000:2.0:1787784867.677050:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:32131:x1874627658254720:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784867.677067:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784867.677084:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1696 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.677118:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.677127:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784867.677131:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784867.677137:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.677524:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 10 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.677541:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 10 pid: 10688 timeout: 758 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784867.677554:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508850f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668440320:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784867.677647:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d540 going to sleep for 11 seconds 00000100:00100000:2.0:1787784867.681894:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508850f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668440320:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784867.681931:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.681951:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:2.0:1787784867.682099:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:32131:x1874627658254720:12345-192.168.203.43@tcp:101:cp.0 Request processed in 5049us (5247us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784867.682107:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 390 00000100:00100000:0.0:1787784867.686507:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784867.687007:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658254848 00000100:00100000:1.0:1787784867.687034:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658254848 00010000:00010000:1.0:1787784867.687108:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 10 pid: 10688 timeout: 758 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784867.687203:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:1.0:1787784867.687215:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658254848:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784867.687235:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784867.687267:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 10 pid: 10688 timeout: 758 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.687289:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 10 pid: 10688 timeout: 758 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.687318:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc7467 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b168f expref: 10 pid: 10688 timeout: 758 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.687339:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784867.687344:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784867.687353:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.687365:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.687415:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784867.687434:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668440448:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784867.687556:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f0979c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784867.699553:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5797:1874627668440448:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784867.699605:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784867.699689:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658254848:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 12474us (13188us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784867.699703:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:0.0:1787784867.710005:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784867.710630:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658255104 00000100:00100000:1.0:1787784867.710643:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658255104 00000100:00100000:1.0:1787784867.710729:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 279 00000100:00100000:1.0:1787784867.710740:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32131:x1874627658255104:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784867.711263:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32131:x1874627658255104:12345-192.168.203.43@tcp:35:cp.0 Request processed in 522us (1269us total) trans 4294968220 rc 0/0 00000100:00100000:1.0:1787784867.711273:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 279 00010000:00010000:3.0:1787784867.761462:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784867.763509:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784867.764214:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658255232 00000100:00100000:3.0:1787784867.764230:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658255232 00000100:00100000:3.0:1787784867.764321:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1386 00000100:00100000:3.0:1787784867.764333:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32134:x1874627658255232:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784867.764362:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784867.764389:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7475 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b169d expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.764477:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.764496:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.764510:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.764540:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765672:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765688:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765843:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765853:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765861:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784867.765914:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00000004:00200000:3.0:1787784867.765920:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784867.765946:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765974:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765985:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.765993:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766618:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766631:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784867.766636:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766646:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766663:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:3.0:1787784867.766666:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.766671:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766678:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc747c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766698:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766705:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766710:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766715:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784867.766718:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766728:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766735:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff9895198b5300) 00010000:00010000:3.0:1787784867.766737:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784867.766740:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766745:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7483 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766801:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7475 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 106 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766818:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 105 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784867.766826:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:3.0:1787784867.766829:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784867.766831:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784867.766947:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32134:x1874627658255232:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2615us (3454us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784867.766958:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1386 00000100:00100000:3.0:1787784867.773565:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784867.774700:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658255360 00000100:00100000:2.0:1787784867.774719:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658255360 00000100:00100000:2.0:1787784867.774808:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:2.0:1787784867.774821:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658255360:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784867.774847:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784867.774871:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc7491 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16a4 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784867.774952:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508850f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668440576:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784867.775314:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d540 going to sleep for 11 seconds 00002000:00010000:2.0:1787784867.783920:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784867.783932:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime: 0 -> 1787784867 00002000:00010000:2.0:1787784867.783935:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb atime: 0 -> 1787784867 00002000:00010000:2.0:1787784867.783938:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime: 0 -> 1787784867 00000100:00100000:2.0:1787784867.783970:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508850f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668440576:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784867.784035:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc7491 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16a4 expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.784067:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc7491 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16a4 expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784867.784084:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403350200, rc 0) 00000100:00100000:2.0:1787784867.784221:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658255360:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 9402us (10686us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784867.784232:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 391 00000100:00100000:0.0:1787784867.801992:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784867.802220:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658255488 00000100:00100000:1.0:1787784867.802235:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658255488 00000100:00100000:1.0:1787784867.802320:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1387 00000100:00100000:1.0:1787784867.802335:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32134:x1874627658255488:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784867.802363:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784867.802391:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7498 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16ab expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.802487:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.802506:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.802519:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784867.802620:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 512 00000004:00200000:1.0:1787784867.802626:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784867.802663:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.802698:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7498 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 107 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.802722:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 106 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.802732:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:1.0:1787784867.802735:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784867.802739:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641600, rc 0) 00000100:00100000:1.0:1787784867.802812:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32134:x1874627658255488:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 480us (839us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784867.802831:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1387 00000100:00100000:3.0:1787784867.810220:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784867.810605:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658255616 00000100:00100000:1.0:1787784867.810617:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658255616 00000100:00100000:1.0:1787784867.810678:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 392 00000100:00100000:1.0:1787784867.810686:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658255616:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784867.810702:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784867.810717:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc74a6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16b2 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784867.810768:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668440704:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784867.810926:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f0979c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784867.816192:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668440704:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784867.816247:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc74a6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16b2 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.816278:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000def92732/0xb0ce63137acc74a6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16b2 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784867.816298:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784867.816828:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658255616:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 5861us (6342us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784867.816840:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 392 00000100:00100000:2.0:1787784867.828375:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784867.828662:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658255872 00000100:00100000:0.0:1787784867.828784:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658255872 00000100:00100000:0.0:1787784867.828872:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1388 00000100:00100000:0.0:1787784867.828885:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32134:x1874627658255872:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:0.0:1787784867.829932:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].41, 136 off13768 00000100:00100000:0.0:1787784867.830378:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32134:x1874627658255872:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 1488us (2006us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784867.830390:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1388 00000040:00080000:2.0:1787784868.145469:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784868.145482:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784868.145528:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 42 count 4 00000040:00080000:2.0:1787784868.145537:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784868.147861:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508851680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668440832:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784868.149325:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051df00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784868.155522:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784868.155818:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658256000 00000100:00100000:3.0:1787784868.155830:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658256000 00000100:00100000:3.0:1787784868.155903:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1389 00000100:00100000:3.0:1787784868.155916:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32144:x1874627658256000:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784868.155967:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784868.156116:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32144:x1874627658256000:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 202us (607us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784868.156125:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1389 00000100:00100000:3.0:1787784868.156296:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508851680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668440832:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784868.160293:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784868.160443:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658256256 00000100:00100000:3.0:1787784868.160457:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658256256 00000100:00100000:3.0:1787784868.160558:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1390 00000100:00100000:3.0:1787784868.160571:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32144:x1874627658256256:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784868.160609:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784868.160638:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc74ad lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16b9 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.160776:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc74b4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.160794:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc74b4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.160811:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc74b4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784868.160823:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:3.0:1787784868.160979:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc74ad lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16b9 expref: 55 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.161216:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc74b4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16b9 expref: 54 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.161230:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:3.0:1787784868.161234:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784868.161239:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784868.161319:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32144:x1874627658256256:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 751us (1044us total) trans 4294968221 rc 0/0 00000100:00100000:3.0:1787784868.161327:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1390 00000100:00100000:0.0:1787784868.165387:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784868.167808:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658256384 00000100:00100000:2.0:1787784868.167828:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658256384 00000100:00100000:2.0:1787784868.167906:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 393 00000100:00100000:2.0:1787784868.167917:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658256384:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784868.167976:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784868.168016:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784868.168028:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784868.168033:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784868.168038:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784868.168086:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784868.168102:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 9 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784868.168124:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508851680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668440960:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784868.168258:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051dcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784868.193407:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784868.193467:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658256512 00000100:00100000:2.0:1787784868.193479:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658256512 00000100:00100000:2.0:1787784868.193565:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 63 00000100:00100000:2.0:1787784868.193578:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658256512:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784868.193612:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784868.193646:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc74b4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16b9 expref: 54 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784868.193674:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:2.0:1787784868.193682:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784868.193750:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2356:x1874627658256512:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 174us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784868.193758:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 63 00000100:00100000:0.0:1787784868.196632:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784868.196730:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508851680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668440960:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784868.196764:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784868.196858:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658256640 00000100:00100000:3.0:1787784868.196867:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658256640 00002000:00100000:3.0:1787784868.196932:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1b:0x0] (0->157) req@ffff989540288000 x1874627658256640/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:749/0 lens 648/0 e 0 to 0 dl 1787784879 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784868.196944:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540288000 with x1874627658256640 ext(0->157) 00010000:00010000:3.0:1787784868.196952:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 10 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.196968:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 10 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784868.196977:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989540288000 00000100:00100000:3.0:1787784868.197023:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 15 00000100:00100000:3.0:1787784868.197035:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658256640:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:3.0:1787784868.197721:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658256640:12345-192.168.203.43@tcp:4:cp.0 Request processed in 682us (1095us total) trans 4294967371 rc 0/0 00000100:00100000:3.0:1787784868.197734:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 15 00002000:00100000:3.0:1787784868.197743:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1b:0x0] (0->157) req@ffff989540288000 x1874627658256640/t4294967371(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:749/0 lens 648/448 e 0 to 0 dl 1787784879 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784868.197754:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540288000 with x1874627658256640 ext(0->157) 00010000:00010000:3.0:1787784868.197762:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 12 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.197777:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 12 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784868.197785:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989540288000 00000100:00100000:3.0:1787784868.205034:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784868.205630:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658256768 00000100:00100000:0.0:1787784868.205644:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658256768 00010000:00010000:0.0:1787784868.205699:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 12 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784868.205760:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 416 00000100:00100000:0.0:1787784868.205771:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658256768:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784868.205790:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784868.205820:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 12 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.205836:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 12 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.205867:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ca94d2c/0xb0ce63137acc746e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1696 expref: 12 pid: 6671 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.205885:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784868.205890:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784868.205897:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.205909:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.205944:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784868.206026:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658256768:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 256us (1020us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784868.206035:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 416 00010000:00010000:0.0:1787784868.206080:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206090:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206121:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206130:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206138:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206169:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784868.206175:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206185:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206198:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.206206:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd6d3335/0xb0ce63137acc74c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784868.206268:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658256384:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 38352us (40893us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784868.206277:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 393 00000100:00100000:2.0:1787784868.213245:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784868.213658:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658257024 00000100:00100000:0.0:1787784868.213677:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658257024 00000100:00100000:0.0:1787784868.213761:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1391 00000100:00100000:0.0:1787784868.213773:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32144:x1874627658257024:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784868.213939:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32144:x1874627658257024:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 168us (709us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784868.213947:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1391 00000100:00100000:0.0:1787784868.220747:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784868.220829:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658257152 00000100:00100000:2.0:1787784868.220842:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658257152 00000100:00100000:2.0:1787784868.220909:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 394 00000100:00100000:2.0:1787784868.220922:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658257152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784868.220944:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784868.220970:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16c0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784868.221019:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 158, mtime: 0x6a8f6ea3, atime: 0x0, ctime: 0x6a8f6ea3, blocks: 0x1 00010000:00010000:2.0:1787784868.221046:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=659s v=(0 0) 00010000:00010000:2.0:1787784868.221051:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784868.221063:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784868.221082:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784868.221098:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16000, rc 0) 00000100:00100000:2.0:1787784868.221212:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658257152:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 292us (481us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784868.221220:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 394 00000100:00100000:2.0:1787784868.226043:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784868.226195:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658257408 00000100:00100000:0.0:1787784868.226208:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658257408 00000100:00100000:0.0:1787784868.226279:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1392 00000100:00100000:0.0:1787784868.226290:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32144:x1874627658257408:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784868.226435:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32144:x1874627658257408:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 147us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784868.226442:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1392 00000100:00100000:3.0:1787784868.231280:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784868.231706:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658257536 00000100:00100000:2.0:1787784868.231720:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658257536 00002000:00100000:2.0:1787784868.231789:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1b:0x0] (0->4095) req@ffff9894030ee580 x1874627658257536/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:749/0 lens 488/0 e 0 to 0 dl 1787784879 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784868.231802:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894030ee580 with x1874627658257536 ext(0->4095) 00002000:00010000:2.0:1787784868.231808:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894030ee580 00000100:00100000:2.0:1787784868.231848:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 119 00000100:00100000:2.0:1787784868.231861:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658257536:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784868.234990:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658257536:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 3113us (3709us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784868.235021:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 119 00002000:00100000:2.0:1787784868.235035:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1b:0x0] (0->4095) req@ffff9894030ee580 x1874627658257536/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:749/0 lens 488/4536 e 0 to 0 dl 1787784879 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784868.235047:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9894030ee580 with x1874627658257536 ext(0->4095) 00002000:00010000:2.0:1787784868.235056:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9894030ee580 00000100:00100000:2.0:1787784868.244479:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784868.245693:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658257664 00000100:00100000:3.0:1787784868.245711:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658257664 00000100:00100000:3.0:1787784868.245784:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1393 00000100:00100000:3.0:1787784868.245795:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32144:x1874627658257664:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784868.245823:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784868.245853:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc74d0 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16c7 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.245897:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.245915:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.245929:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.246085:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc74d0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.246110:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 54 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.246120:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:3.0:1787784868.246123:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784868.246127:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784868.246244:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32144:x1874627658257664:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 449us (1785us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784868.246252:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1393 00000100:00100000:3.0:1787784868.256710:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784868.256880:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658257792 00000100:00100000:2.0:1787784868.256895:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658257792 00000100:00100000:2.0:1787784868.256976:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1394 00000100:00100000:2.0:1787784868.256989:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32144:x1874627658257792:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784868.257049:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784868.257215:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32144:x1874627658257792:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 227us (521us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784868.257224:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1394 00000100:00100000:3.0:1787784868.265535:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784868.266251:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658257920 00000100:00100000:1.0:1787784868.266265:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658257920 00000100:00100000:1.0:1787784868.266342:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:1.0:1787784868.266354:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658257920:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784868.266417:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266442:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784868.266447:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784868.266454:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266471:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266512:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266522:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266537:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266555:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784868.266561:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266572:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266588:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.266747:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2606829/0xb0ce63137acc74de lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784868.266852:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658257920:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 500us (1332us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784868.266861:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 395 00000100:00100000:2.0:1787784868.300103:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784868.301266:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658258176 00000100:00100000:1.0:1787784868.301283:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658258176 00000100:00100000:1.0:1787784868.301385:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1395 00000100:00100000:1.0:1787784868.301398:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32144:x1874627658258176:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784868.301509:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.301533:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 54 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.301544:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.301590:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 54 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.301607:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 54 pid: 5811 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784868.301624:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668441088:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784868.301765:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953f0979c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784868.323721:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784868.323851:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658258304 00000100:00100000:3.0:1787784868.323863:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658258304 00010000:00010000:3.0:1787784868.323919:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 55 pid: 5811 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784868.323971:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 417 00000100:00100000:3.0:1787784868.323983:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:2356:x1874627658258304:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784868.324002:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784868.324020:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 55 pid: 5811 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.324037:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 55 pid: 5811 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.324067:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:3.0:1787784868.324073:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.324086:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.324095:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.324120:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784868.324221:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:2356:x1874627658258304:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 240us (516us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784868.324230:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 417 00010000:00010000:1.0:1787784868.325986:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc74d7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c7 expref: 54 pid: 5811 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784868.326038:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668441088:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784868.326072:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326084:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326090:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326428:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326438:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326445:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326452:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784868.326457:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326466:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326483:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:1.0:1787784868.326486:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784868.326491:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784868.326497:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc74e5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784868.326927:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784868.326934:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784868.327477:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1787784868.327490:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784868.328188:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32144:x1874627658258176:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 26782us (28087us total) trans 4294968222 rc 0/0 00000100:00100000:1.0:1787784868.328205:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1395 00000100:00100000:3.0:1787784868.336219:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784868.336354:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658258560 00000100:00100000:2.0:1787784868.336367:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658258560 00000100:00100000:2.0:1787784868.336437:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 280 00000100:00100000:2.0:1787784868.336449:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32144:x1874627658258560:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784868.336791:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32144:x1874627658258560:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 343us (588us total) trans 4294968223 rc 0/0 00000100:00100000:2.0:1787784868.336804:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 280 00000001:00080000:3.0:1787784868.400187:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967371 is committed 00010000:00010000:0.0:1787784868.786829:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784868.886894:0:22127:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784868.886908:0:22127:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784868.886970:0:22127:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 42 count 4 00000040:00080000:1.0:1787784868.886977:0:22127:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:2.0:1787784868.931565:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968222 is committed 00000001:00080000:2.0:1787784868.931584:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968223 is committed 00000001:00080000:2.0:1787784868.931595:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968218 is committed 00000020:00080000:2.0:1787784868.931608:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 32 batchid updated 00000020:00080000:2.0:1787784868.931630:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 32 00000020:00080000:2.0:1787784868.931644:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 32 00000100:00100000:1.0:1787784868.945389:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784868.949233:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658258688 00000100:00100000:0.0:1787784868.949256:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658258688 00000100:00100000:0.0:1787784868.949359:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:0.0:1787784868.949372:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658258688:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784868.949403:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784868.949428:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16ce expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.949464:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784868.949469:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784868.949480:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.949503:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.949519:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b800, rc 0) 00000100:00100000:0.0:1787784868.949612:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658258688:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 241us (4234us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784868.949620:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 396 00000100:00100000:3.0:1787784868.957631:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784868.958055:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658258944 00000100:00100000:0.0:1787784868.958069:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658258944 00000100:00100000:0.0:1787784868.958178:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1396 00000100:00100000:0.0:1787784868.958191:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32145:x1874627658258944:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784868.958220:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784868.958642:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc74f3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16d5 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.958798:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc74fa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.958817:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc74fa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.958833:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc74fa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784868.958848:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:0.0:1787784868.959095:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc74f3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16d5 expref: 108 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.959130:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc74fa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16d5 expref: 107 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.959142:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784868.963608:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784868.963630:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150be00, rc 0) 00000100:00100000:0.0:1787784868.963768:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32145:x1874627658258944:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 5573us (6148us total) trans 4294968224 rc 0/0 00000100:00100000:0.0:1787784868.963782:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1396 00000100:00100000:3.0:1787784868.970242:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784868.973101:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658259072 00000100:00100000:0.0:1787784868.973123:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658259072 00000100:00100000:0.0:1787784868.973243:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 397 00000100:00100000:0.0:1787784868.973269:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658259072:12345-192.168.203.43@tcp:1:lfs.0 00010000:00010000:0.0:1787784868.973329:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.973356:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 8 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.973367:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.973377:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784868.973381:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784868.973386:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.973434:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.973451:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 17 pid: 6671 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784868.973470:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668441216:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784868.973611:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 8 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.973622:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 8 pid: 6670 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784868.973640:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668441344:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784868.973666:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507d673c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784868.980184:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668441216:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784868.980208:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668441344:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784868.980252:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784868.981749:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784868.981798:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784868.981833:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658259200 00000100:00100000:3.0:1787784868.981844:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658259200 00010000:00010000:3.0:1787784868.981893:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 18 pid: 6671 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784868.981934:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658259328 00000100:00100000:3.0:1787784868.981938:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658259328 00010000:00010000:3.0:1787784868.981952:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 9 pid: 6670 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784868.981976:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 418 00000100:00100000:3.0:1787784868.981987:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c95850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2357:x1874627658259200:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784868.982018:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1787784868.982044:0:5796:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:3.0:1787784868.982051:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 18 pid: 6671 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982065:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 18 pid: 6671 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982092:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acbc6c6b/0xb0ce63137acc74ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ce expref: 18 pid: 6671 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982112:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784868.982195:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c95850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658259200:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 208us (470us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784868.982204:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 418 00000100:00100000:3.0:1787784868.982264:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 419 00000100:00100000:3.0:1787784868.982271:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658259328:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784868.982280:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784868.982290:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 9 pid: 6670 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982306:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 9 pid: 6670 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982320:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa7f5466/0xb0ce63137acc74c9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16c0 expref: 9 pid: 6670 timeout: 760 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982331:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784868.982335:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784868.982342:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982353:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784868.982381:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784868.982402:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658259328:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 132us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784868.982407:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 419 00010000:00010000:0.0:1787784868.988490:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988517:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988566:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988575:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988586:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988595:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784868.988601:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988611:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988635:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784868.988644:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7508 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784868.988771:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658259072:12345-192.168.203.43@tcp:1:lfs.0 Request processed in 15511us (18536us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784868.988783:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 397 00000100:00100000:2.0:1787784868.999964:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784869.000257:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658259584 00000100:00100000:0.0:1787784869.000271:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658259584 00000100:00100000:0.0:1787784869.000350:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 281 00000100:00100000:0.0:1787784869.000362:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32145:x1874627658259584:12345-192.168.203.43@tcp:35:lfs.0 00000004:00010000:0.0:1787784869.000448:0:5815:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x2a:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc74fa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16d5 expref: 107 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784869.000795:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968225 00000001:00080000:0.0:1787784869.001579:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968226 00010000:00010000:0.0:1787784869.001658:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.001680:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 49 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.001691:0:5815:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 110 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.001702:0:5815:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 5/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.001748:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 110 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.001767:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 110 pid: 5812 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784869.001785:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8b40 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668441472:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784869.001914:0:5815:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 49 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.001925:0:5815:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 49 pid: 5813 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784869.001935:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668441600:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784869.001960:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784869.013096:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784869.013217:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784869.013260:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658259712 00000100:00100000:3.0:1787784869.013272:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658259712 00000100:00100000:0.0:1787784869.013342:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658259840 00010000:00010000:3.0:1787784869.013343:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 111 pid: 5812 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784869.013347:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658259840 00010000:00010000:0.0:1787784869.013369:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 50 pid: 5813 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784869.013382:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:3.0:1787784869.013394:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658259712:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784869.013410:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784869.013418:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 111 pid: 5812 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.013432:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 111 pid: 5812 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.013460:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:3.0:1787784869.013466:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.013475:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.013482:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784869.013540:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658259712:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 148us (461us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784869.017958:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:3.0:1787784869.018265:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 421 00000100:00100000:3.0:1787784869.018291:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2356:x1874627658259840:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784869.018323:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784869.018338:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 50 pid: 5813 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.018359:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 50 pid: 5813 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.018399:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:3.0:1787784869.018406:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.018416:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.018427:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784869.018453:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784869.018543:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2356:x1874627658259840:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 266us (5325us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784869.018552:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 421 00000100:00080000:2.0:1787784869.021073:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784869.021186:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc748a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b169d expref: 110 pid: 5812 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784869.021216:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8b40 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668441472:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784869.021234:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc7459 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1688 expref: 49 pid: 5813 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784869.021246:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5815:1874627668441600:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784869.021286:0:5815:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784869.021296:0:5815:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784869.021302:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784869.021465:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968227 00000001:00080000:2.0:1787784869.022636:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968228 00000001:00080000:2.0:1787784869.022837:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968229 00000100:00100000:2.0:1787784869.022950:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668441728:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1787784869.023054:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784869.023181:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784869.023678:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668441728 00000100:00100000:3.0:1787784869.023687:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668441728 00000100:00100000:3.0:1787784869.023909:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 69 00000100:00100000:3.0:1787784869.023921:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668441728:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1787784869.024049:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668441728:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 129us (994us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784869.024058:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 69 00000100:00100000:3.0:1787784869.024124:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668441728:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:3.0:1787784869.024568:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968230 00000100:00100000:3.0:1787784869.024646:0:5815:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668441856:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1787784869.024688:0:5815:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784869.024918:0:5815:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a56c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784869.025115:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668441856 00000100:00100000:0.0:1787784869.025123:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668441856 00000100:00100000:0.0:1787784869.025186:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 70 00000100:00100000:0.0:1787784869.025197:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5815:x1874627668441856:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1787784869.025470:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5815:x1874627668441856:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 275us (782us total) trans 4294967372 rc 0/0 00000100:00100000:0.0:1787784869.025478:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 70 00000100:00100000:0.0:1787784869.025990:0:5815:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5815:1874627668441856:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:0.0:1787784869.026043:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.026056:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.026066:0:5815:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.026074:0:5815:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784869.026079:0:5815:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.026089:0:5815:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.026121:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784869.026125:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1787784869.026132:0:5815:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784869.026173:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc750f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784869.026456:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x50:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1787784869.026589:0:5815:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].13, 64 off9048 00000004:00080000:0.0:1787784869.026602:0:5815:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 660113454304ns, next at 661113451824ns, rc = 0 00000001:00080000:0.0:1787784869.026612:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968231 00010000:00010000:0.0:1787784869.026666:0:5815:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989520131600) 00010000:00010000:0.0:1787784869.026670:0:5815:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787784869.026678:0:5815:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc74fa lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16d5 expref: 114 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784869.027307:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:32145:x1874627658259584:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 26942us (27352us total) trans 4294968232 rc 0/0 00000100:00100000:0.0:1787784869.027320:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 281 00000020:00080000:0.0:1787784869.027326:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00080000:2.0:1787784869.028540:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784869.361746:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540666d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668441984:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784869.361843:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784869.362173:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668441984 00000100:00100000:2.0:1787784869.362184:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668441984 00000100:00100000:2.0:1787784869.362239:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1787784869.362251:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668441984:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784869.362282:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3854565376 00000100:00100000:2.0:1787784869.362472:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668441984:12345-0@lo:13:osp-pre-0-0.0 Request processed in 222us (628us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784869.362481:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:1.0:1787784869.362615:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540666d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668441984:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784869.809606:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784869.889489:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784869.890099:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658260096 00000100:00100000:1.0:1787784869.890110:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658260096 00000100:00100000:1.0:1787784869.890336:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1397 00000100:00100000:1.0:1787784869.890348:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:32155:x1874627658260096:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784869.890372:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784869.890444:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7516 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16dc expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.890515:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.890530:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.890680:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.890698:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.890706:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.890712:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891123:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc752b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891132:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc752b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891140:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc752b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784869.891167:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:1.0:1787784869.891270:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891277:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891283:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891288:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784869.891292:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891299:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891310:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:1.0:1787784869.891313:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784869.891317:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891322:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc751d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891333:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891342:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891346:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891350:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784869.891352:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891358:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891363:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff989531111e00) 00010000:00010000:1.0:1787784869.891366:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784869.891369:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.891375:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7524 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.892397:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7516 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16dc expref: 116 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.892924:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc752b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16dc expref: 115 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.892934:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:1.0:1787784869.892937:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784869.892941:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641400, rc 0) 00000100:00100000:1.0:1787784869.893501:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:32155:x1874627658260096:12345-192.168.203.43@tcp:101:cat.0 Request processed in 3152us (4016us total) trans 4294968233 rc 0/0 00000100:00100000:1.0:1787784869.893512:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1397 00000100:00100000:0.0:1787784869.906266:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784869.906403:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658260352 00000100:00100000:1.0:1787784869.906414:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658260352 00000100:00100000:1.0:1787784869.906475:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1398 00000100:00100000:1.0:1787784869.906486:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:32154:x1874627658260352:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784869.906511:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784869.906532:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7539 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16e3 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.906641:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.906656:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.906668:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784869.906690:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:1.0:1787784869.906811:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7539 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 118 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.906828:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 117 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784869.906837:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:1.0:1787784869.906841:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784869.906845:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784869.906902:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32154:x1874627658260352:12345-192.168.203.43@tcp:101:cat.0 Request processed in 418us (645us total) trans 4294968234 rc 0/0 00000100:00100000:1.0:1787784869.906911:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1398 00000100:00100000:0.0:1787784869.911748:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784869.911842:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784869.912078:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658260608 00000100:00100000:2.0:1787784869.912089:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658260608 00000100:00100000:2.0:1787784869.912181:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 64 00000100:00100000:2.0:1787784869.912192:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32155:x1874627658260608:12345-192.168.203.43@tcp:103:cat.0 00010000:00010000:2.0:1787784869.912210:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784869.912434:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc752b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16dc expref: 117 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784869.912453:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:2.0:1787784869.912458:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784869.912516:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:32155:x1874627658260608:12345-192.168.203.43@tcp:103:cat.0 Request processed in 326us (777us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784869.912527:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 64 00000100:00100000:3.0:1787784869.912573:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658260864 00000100:00100000:3.0:1787784869.912580:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658260864 00000100:00100000:3.0:1787784869.912634:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 282 00000100:00100000:3.0:1787784869.912644:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:32154:x1874627658260864:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784869.912902:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32154:x1874627658260864:12345-192.168.203.43@tcp:35:cat.0 Request processed in 260us (1060us total) trans 4294968235 rc 0/0 00000100:00100000:3.0:1787784869.912912:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 282 00000100:00100000:0.0:1787784869.917903:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784869.917991:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658260992 00000100:00100000:2.0:1787784869.918011:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658260992 00000100:00100000:2.0:1787784869.918068:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1399 00000100:00100000:2.0:1787784869.918078:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32155:x1874627658260992:12345-192.168.203.43@tcp:58:cat.0 00000004:00020000:2.0:1787784869.918190:0:5813:0:(mdt_hsm_cdt_client.c:349:mdt_hsm_register_hal()) cfs_race id 18b sleeping 00010000:00010000:3.0:1787784870.834672:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784871.409381:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784871.409405:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668442112:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784871.409475:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784871.409652:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784871.409662:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668442240:0@lo:400:kworker.0 00000100:00100000:1.0:1787784871.409684:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668442496:0@lo:400:kworker.0 00000100:00100000:2.0:1787784871.409709:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784871.409723:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784871.409749:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668442624:0@lo:400:kworker.0 00000100:00100000:3.0:1787784871.409776:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668442752:0@lo:400:kworker.0 00000100:00100000:2.0:1787784871.409776:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784871.409802:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784871.409806:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668442240 00000100:00100000:2.0:1787784871.409813:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668442240 00000100:00100000:3.0:1787784871.409829:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668442496 00000100:00100000:3.0:1787784871.409831:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668442496 00000100:00100000:2.0:1787784871.409854:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:3.0:1787784871.409859:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668442752 00000100:00100000:2.0:1787784871.409861:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668442240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784871.409862:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668442752 00000100:00100000:3.0:1787784871.409879:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:3.0:1787784871.409884:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668442496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784871.409909:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668442240:12345-0@lo:400:kworker.0 Request processed in 48us (199us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784871.409912:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668442496:12345-0@lo:400:kworker.0 Request processed in 28us (189us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784871.409917:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:3.0:1787784871.409918:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:3.0:1787784871.409932:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1787784871.409937:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668442752:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787784871.409944:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784871.409997:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668442240:0@lo:400:kworker.0 00000100:00100000:3.0:1787784871.410233:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668442752:12345-0@lo:400:kworker.0 Request processed in 295us (429us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784871.410240:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:2.0:1787784871.410272:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668442752:0@lo:400:kworker.0 00000100:00100000:2.0:1787784871.410300:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668442624 00000100:00100000:2.0:1787784871.410304:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668442624 00000100:00100000:2.0:1787784871.410347:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:2.0:1787784871.410354:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668442624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784871.410391:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668442624:12345-0@lo:400:kworker.0 Request processed in 38us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784871.410398:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:2.0:1787784871.410424:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668442624:0@lo:400:kworker.0 00000100:00100000:0.0:1787784871.410494:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668442112 00000100:00100000:0.0:1787784871.410499:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668442112 00000100:00080000:3.0:1787784871.410631:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784871.410634:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784871.410655:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668442368:0@lo:400:kworker.0 00000100:00100000:3.0:1787784871.410683:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784871.410739:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668442368 00000100:00100000:3.0:1787784871.410743:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668442368 00000100:00100000:3.0:1787784871.410765:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:3.0:1787784871.410772:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668442368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784871.410806:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668442368:12345-0@lo:400:kworker.0 Request processed in 35us (122us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784871.410812:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:3.0:1787784871.410837:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668442368:0@lo:400:kworker.0 00000100:00100000:0.0:1787784871.411289:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1787784871.411300:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668442112:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784871.411328:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784871.411381:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668442112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (1905us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784871.411388:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:1.0:1787784871.411411:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668442496:0@lo:400:kworker.0 00000100:00100000:3.0:1787784871.411464:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668442112:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1787784871.857424:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784872.433461:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784872.433470:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784872.433648:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 42 count 2 00000040:00080000:3.0:1787784872.433654:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784872.713754:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784872.713801:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784872.713835:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784872.713865:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784872.713886:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784872.713904:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784872.713933:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784872.713963:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658261376 00000100:00100000:0.0:1787784872.713972:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658261376 00000100:00100000:3.0:1787784872.714142:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658261120 00000100:00100000:3.0:1787784872.714178:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658261120 00000100:00100000:3.0:1787784872.714223:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 607 00000100:00100000:3.0:1787784872.714234:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658261120:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784872.714301:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658261120:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 69us (558us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784872.714308:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 607 00000100:00100000:0.0:1787784872.714477:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658261888 00000100:00100000:0.0:1787784872.714481:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658261888 00000100:00100000:2.0:1787784872.714586:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658261632 00000100:00100000:2.0:1787784872.714591:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658261632 00000100:00100000:2.0:1787784872.714618:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658261248 00000100:00100000:2.0:1787784872.714621:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658261248 00000100:00100000:2.0:1787784872.714643:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 474 00000100:00100000:2.0:1787784872.714650:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2355:x1874627658261632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784872.714688:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2355:x1874627658261632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (852us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784872.714695:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 474 00000100:00100000:2.0:1787784872.714708:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 475 00000100:00100000:2.0:1787784872.714713:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:2355:x1874627658261248:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784872.714734:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:2355:x1874627658261248:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (848us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784872.714740:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 475 00000100:00100000:3.0:1787784872.720295:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658261504 00000100:00100000:3.0:1787784872.720311:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658261504 00000100:00100000:3.0:1787784872.720363:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658261760 00000100:00100000:3.0:1787784872.720366:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658261760 00000100:00100000:3.0:1787784872.720387:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:3.0:1787784872.720397:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658261376:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784872.720493:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658261376:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 97us (6692us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784872.720510:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:3.0:1787784872.720757:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:3.0:1787784872.720765:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658261888:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784872.720794:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658261888:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (6928us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784872.720800:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:3.0:1787784872.720813:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 442 00000100:00100000:3.0:1787784872.720819:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658261504:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784872.720841:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:2355:x1874627658261504:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (6936us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784872.720847:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 442 00000100:00100000:3.0:1787784872.720859:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 443 00000100:00100000:3.0:1787784872.720864:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658261760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784872.720886:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658261760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (6952us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784872.720892:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 443 00000100:00080000:3.0:1787784872.720923:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784872.720929:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784872.720932:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784872.883337:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784873.530056:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967372 is committed 00010000:00010000:3.0:1787784873.907411:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787784874.044651:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968232 is committed 00000004:00080000:1.0:1787784874.044665:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 15 committed 00000100:00100000:3.0:1787784874.044975:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540666d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668442880:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784874.045054:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:0.0:1787784874.045659:0:22194:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968234 is committed 00000001:00080000:0.0:1787784874.045700:0:22194:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968235 is committed 00000020:00080000:0.0:1787784874.045708:0:22194:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98953efb1000 refcount 2 committed 0 result 0 batchid 33 00000020:00080000:0.0:1787784874.045713:0:22194:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989508cffa00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98953efb1b40 00000020:00080000:0.0:1787784874.045716:0:22194:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989508cff200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989401d17000 00000020:00080000:0.0:1787784874.045733:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 33 committed batchid 32 00000020:00080000:0.0:1787784874.045903:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 32 00000100:00100000:3.0:1787784874.046413:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668442880 00000100:00100000:3.0:1787784874.046422:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668442880 00000100:00100000:3.0:1787784874.046487:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1787784874.046497:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668442880:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787784874.046526:0:6671:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:27 count 1 00002000:00010000:3.0:1787784874.046562:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 158, mtime: 0x6a8f6ea3, atime: 0x0, ctime: 0x6a8f6ea3, blocks: 0xb 00010000:00010000:3.0:1787784874.046570:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.046586:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=665s v=(0 0) 00010000:00010000:3.0:1787784874.046591:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784874.046597:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.046613:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.046623:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.046631:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.046657:0:6671:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784874.047398:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668442880:12345-0@lo:6:osp-syn-0-0.0 Request processed in 900us (2342us total) trans 4294967373 rc 0/0 00000100:00100000:3.0:1787784874.047408:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00010000:00010000:3.0:1787784874.047440:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784874.047447:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.047457:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784874.047461:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.047471:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.047488:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.047495:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc754e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784874.047513:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784874.047515:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784874.047516:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784874.047543:0:3278:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967372 gen 2 00000004:00080000:3.0:1787784874.047548:0:3278:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967372 committed transno 4294967372 00000100:00100000:3.0:1787784874.047569:0:3278:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff989540666d00 x1874627668442880/t4294967373(4294967373) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1787784890 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:3.0:1787784874.047857:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989540666d00/2, rc 0, transno 4294967373 00000100:00100000:3.0:1787784874.047865:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540666d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668442880:0@lo:6:osp-syn-0-0.0 00000100:00080000:0.0:1787784874.047917:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784874.481581:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784874.481604:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668443008:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784874.481668:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784874.483320:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668443008 00000100:00100000:1.0:1787784874.483333:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668443008 00000100:00100000:1.0:1787784874.483431:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1787784874.483443:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668443008:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784874.483479:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784874.483546:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668443008:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (1877us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784874.483553:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1787784874.483571:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668443008:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784874.929309:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00020000:2.0:1787784874.993264:0:5813:0:(mdt_hsm_cdt_client.c:349:mdt_hsm_register_hal()) cfs_fail_race id 18b awake: rc=0 00010000:00010000:2.0:1787784875.001069:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.001095:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 105 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.001111:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.001351:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 105 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.001366:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 105 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.001382:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668443136:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784875.001481:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd240 going to sleep for 11 seconds 00000100:00100000:3.0:1787784875.007319:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668443136:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784875.007348:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.008032:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.008400:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658262016 00000100:00100000:2.0:1787784875.008407:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658262016 00010000:00010000:2.0:1787784875.008447:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 106 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.008481:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 422 00000100:00100000:2.0:1787784875.008491:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:2357:x1874627658262016:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784875.008505:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784875.008512:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 106 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.008524:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 106 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.008544:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7540 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16e3 expref: 106 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.008562:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:2.0:1787784875.008567:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.008576:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.008583:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.008603:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784875.008644:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:2357:x1874627658262016:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 155us (616us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784875.008652:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 422 00000100:00080000:0.0:1787784875.008727:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784875.010539:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.010555:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784875.011022:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].42, 136 off13904 00000100:00100000:3.0:1787784875.011110:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32155:x1874627658260992:12345-192.168.203.43@tcp:58:cat.0 Request processed in 5093030us (5093211us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.011120:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1399 00000100:00100000:3.0:1787784875.015486:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.015562:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658262144 00000100:00100000:2.0:1787784875.015572:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658262144 00000100:00100000:2.0:1787784875.015625:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1400 00000100:00100000:2.0:1787784875.015635:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32155:x1874627658262144:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784875.015655:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784875.015687:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16ea expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.016420:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.016435:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 4/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.016460:0:10850:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.016473:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.016486:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:2.0:1787784875.016489:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784875.016493:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.016501:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:2.0:1787784875.016564:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32155:x1874627658262144:12345-192.168.203.43@tcp:101:cat.0 Request processed in 930us (1087us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784875.016571:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1400 00000040:00080000:3.0:1787784875.505628:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784875.505636:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784875.505672:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 43 count 3 00000040:00080000:3.0:1787784875.505677:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784875.505804:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ec780 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668443264:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784875.505909:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504c7ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784875.512086:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ec780 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668443264:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784875.514545:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.514757:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658262272 00000100:00100000:0.0:1787784875.514765:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658262272 00000100:00100000:0.0:1787784875.514831:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1401 00000100:00100000:0.0:1787784875.514840:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32157:x1874627658262272:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784875.514868:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784875.514968:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32157:x1874627658262272:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 129us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784875.514976:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1401 00000100:00080000:3.0:1787784875.515178:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784875.525593:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.525688:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658262400 00000100:00100000:0.0:1787784875.525698:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658262400 00000100:00100000:0.0:1787784875.525755:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1402 00000100:00100000:0.0:1787784875.525765:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32157:x1874627658262400:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1787784875.525834:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525852:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525864:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525880:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525887:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525894:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525917:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525925:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.525931:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784875.525976:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00000004:00200000:0.0:1787784875.525981:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784875.525994:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526004:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526012:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526017:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526024:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784875.526027:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526035:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526047:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:0.0:1787784875.526050:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784875.526055:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526061:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc7563 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526074:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526078:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526083:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526088:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784875.526090:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.526096:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527325:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x98b61b69 (ffff989520108300) 00010000:00010000:0.0:1787784875.527329:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784875.527335:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527342:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351400/0xb0ce63137acc756a lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527367:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527374:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527379:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527384:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403350200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784875.527388:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527394:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527402:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:0.0:1787784875.527404:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784875.527407:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.527412:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7571 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784875.527503:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32157:x1874627658262400:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 1737us (1916us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784875.527512:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1402 00000100:00100000:2.0:1787784875.546424:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784875.546645:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658262656 00000100:00100000:1.0:1787784875.546655:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658262656 00000100:00100000:1.0:1787784875.546721:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1403 00000100:00100000:1.0:1787784875.546731:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32157:x1874627658262656:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784875.546750:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784875.546779:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7578 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16f1 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.546880:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.546896:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.546907:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784875.546917:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:1.0:1787784875.547063:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547070:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547077:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547083:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.547086:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547094:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547105:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:1.0:1787784875.547108:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784875.547112:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547118:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc757f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547209:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7578 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16f1 expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547221:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784875.547223:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784875.547227:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7578 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16f1 expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.547238:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784875.547621:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32157:x1874627658262656:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 890us (1205us total) trans 4294968236 rc 301/301 00000100:00100000:1.0:1787784875.547629:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1403 00000100:00100000:3.0:1787784875.562590:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784875.563554:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658262784 00000100:00100000:1.0:1787784875.563571:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658262784 00000100:00100000:1.0:1787784875.563666:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1404 00000100:00100000:1.0:1787784875.563680:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539378780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32157:x1874627658262784:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784875.563730:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784875.564552:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539378780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32157:x1874627658262784:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 869us (1971us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784875.564565:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1404 00000100:00100000:3.0:1787784875.579988:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.587449:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658263040 00000100:00100000:0.0:1787784875.587472:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658263040 00000100:00100000:0.0:1787784875.587635:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1405 00000100:00100000:0.0:1787784875.587788:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32157:x1874627658263040:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784875.587830:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784875.587872:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc758d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16f8 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.604638:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7594 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.604678:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7594 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.604694:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7594 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784875.604713:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2b:0x0]: returning size 0 00010000:00010000:2.0:1787784875.604805:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc758d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16f8 expref: 53 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.604840:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7594 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16f8 expref: 52 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.604861:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff989520131100) 00010000:00010000:2.0:1787784875.604865:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784875.604869:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16400, rc 0) 00000100:00100000:2.0:1787784875.605354:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32157:x1874627658263040:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 17705us (25379us total) trans 4294968237 rc 0/0 00000100:00100000:2.0:1787784875.605368:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1405 00000100:00100000:2.0:1787784875.622645:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.627426:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658263296 00000100:00100000:0.0:1787784875.627450:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658263296 00000100:00100000:1.0:1787784875.628128:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1406 00000100:00100000:1.0:1787784875.628144:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32157:x1874627658263296:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784875.628264:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784875.628302:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7594 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16f8 expref: 52 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.628329:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff989520131100) 00010000:00010000:1.0:1787784875.628342:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784875.628376:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.628393:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 104 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.628403:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.628447:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 104 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.628462:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 104 pid: 10850 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.628479:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668443392:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784875.628637:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff98950310dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784875.646324:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.646487:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658263424 00000100:00100000:0.0:1787784875.646496:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658263424 00010000:00010000:0.0:1787784875.646544:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 105 pid: 10850 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784875.646585:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 423 00000100:00100000:0.0:1787784875.646594:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:2357:x1874627658263424:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784875.646609:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784875.646616:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 105 pid: 10850 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.646629:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 105 pid: 10850 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.646650:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:0.0:1787784875.646657:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.646666:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.646691:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.646712:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784875.646791:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:2357:x1874627658263424:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 197us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784875.646798:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 423 00010000:00010000:3.0:1787784875.647321:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc749f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ab expref: 104 pid: 10850 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.647352:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668443392:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784875.647378:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647387:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647393:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647414:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647424:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647432:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647813:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647822:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647828:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647835:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784875.647839:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647847:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647858:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:3.0:1787784875.647861:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784875.647865:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647871:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc75a2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647882:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647888:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647892:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647897:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784875.647900:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647905:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647917:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xd6d04830 (ffff9895198b5e00) 00010000:00010000:3.0:1787784875.647919:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784875.647922:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.647927:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75a9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xd6d04830 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.648033:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32157:x1874627658263296:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 19892us (25402us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784875.648042:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1406 00000100:00100000:1.0:1787784875.658501:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.660248:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658263552 00000100:00100000:3.0:1787784875.660269:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658263552 00000100:00100000:3.0:1787784875.660872:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1407 00000100:00100000:3.0:1787784875.660884:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32157:x1874627658263552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784875.660906:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784875.660930:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75b0 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b16ff expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.660980:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.660994:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.661006:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.661118:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75b0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.661137:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.661184:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:3.0:1787784875.661189:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784875.661192:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784875.661287:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32157:x1874627658263552:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 400us (2792us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.661295:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1407 00000100:00100000:0.0:1787784875.674398:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.674960:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658263808 00000100:00100000:2.0:1787784875.674971:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658263808 00000100:00100000:2.0:1787784875.675040:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1408 00000100:00100000:2.0:1787784875.675051:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32157:x1874627658263808:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784875.675138:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.675212:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.675224:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784875.675643:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006c8627e1 reply state 000000005d39e7e2 transno 4294968238 ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.675654:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784875.675751:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32157:x1874627658263808:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 700us (1362us total) trans 4294968238 rc 0/0 00000100:00100000:2.0:1787784875.675757:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1408 00000100:00100000:2.0:1787784875.683444:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.683528:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658263936 00000100:00100000:3.0:1787784875.683546:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658263936 00000100:00100000:3.0:1787784875.683611:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1409 00000100:00100000:3.0:1787784875.683623:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32157:x1874627658263936:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784875.683651:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784875.683674:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1706 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.683739:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 55 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.683754:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 55 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.683765:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:3.0:1787784875.683768:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784875.683772:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:3.0:1787784875.683836:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32157:x1874627658263936:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 215us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.683844:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1409 00010000:00010000:1.0:1787784875.684076:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.684087:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.684093:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.684099:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.684103:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.684111:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.684122:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:1.0:1787784875.684124:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787784875.684127:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.684132:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75be lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.688631:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.688701:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658264064 00000100:00100000:2.0:1787784875.688710:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658264064 00000100:00100000:2.0:1787784875.688764:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 283 00000100:00100000:2.0:1787784875.688773:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32157:x1874627658264064:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 00000100:00100000:2.0:1787784875.688912:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32157:x1874627658264064:12345-192.168.203.43@tcp:49:lhsmtool_posix.0 Request processed in 139us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784875.688921:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 283 00000100:00100000:0.0:1787784875.691496:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.691683:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658264320 00000100:00100000:2.0:1787784875.691693:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658264320 00000100:00100000:2.0:1787784875.691747:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00100000:2.0:1787784875.691758:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32157:x1874627658264320:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784875.692008:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32157:x1874627658264320:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 251us (520us total) trans 4294968239 rc 0/0 00000100:00100000:2.0:1787784875.692016:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 284 00000100:00100000:1.0:1787784875.694026:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.694339:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658264576 00000100:00100000:3.0:1787784875.694347:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658264576 00000100:00100000:3.0:1787784875.694404:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1410 00000100:00100000:3.0:1787784875.694412:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32157:x1874627658264576:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784875.694519:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32157:x1874627658264576:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 107us (500us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.694527:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1410 00000100:00100000:1.0:1787784875.701921:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.705741:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658264832 00000100:00100000:2.0:1787784875.705755:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658264832 00000100:00100000:2.0:1787784875.705829:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1411 00000100:00100000:2.0:1787784875.705840:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32157:x1874627658264832:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784875.705862:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784875.705885:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75cc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b170d expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.706005:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.706022:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 56 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.706030:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.706064:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 56 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.706078:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 56 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.706092:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668443520:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784875.706202:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd480 going to sleep for 11 seconds 00000100:00100000:1.0:1787784875.709591:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.709963:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658264960 00000100:00100000:3.0:1787784875.709973:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658264960 00010000:00010000:3.0:1787784875.710013:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 57 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.710056:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 424 00000100:00100000:3.0:1787784875.710067:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c95c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+57:2356:x1874627658264960:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784875.710082:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784875.710088:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 57 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.710099:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 57 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.710121:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:3.0:1787784875.710126:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.710135:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.710143:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.710194:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784875.710238:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c95c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+57:2356:x1874627658264960:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 172us (656us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.710245:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 424 00010000:00010000:2.0:1787784875.710528:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc75c5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1706 expref: 56 pid: 20438 timeout: 766 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.710558:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668443520:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784875.710577:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.710586:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.710596:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784875.710614:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2b:0x0]: returning size 0 00000004:00080000:2.0:1787784875.710918:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1e:0x0], index 1: 0 00010000:00010000:2.0:1787784875.711071:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.711081:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.711087:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.711093:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784875.711097:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.711105:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.711115:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:2.0:1787784875.711118:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784875.711122:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.711128:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75d3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.713389:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75cc lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b170d expref: 56 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.713414:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff9895198b5700) 00010000:00010000:2.0:1787784875.713417:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784875.713422:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc75cc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b170d expref: 56 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.713438:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16400, rc 0) 00000100:00100000:2.0:1787784875.713628:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32157:x1874627658264832:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 7786us (11714us total) trans 4294968240 rc 301/301 00000100:00100000:2.0:1787784875.713637:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1411 00000100:00100000:1.0:1787784875.729209:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.729304:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658265216 00000100:00100000:0.0:1787784875.729315:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658265216 00000100:00100000:0.0:1787784875.729368:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:0.0:1787784875.729378:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32157:x1874627658265216:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784875.729730:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32157:x1874627658265216:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 352us (527us total) trans 4294968241 rc 0/0 00000100:00100000:0.0:1787784875.729738:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 285 00000100:00100000:0.0:1787784875.732354:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.732497:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658265344 00000100:00100000:2.0:1787784875.732508:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658265344 00000100:00100000:2.0:1787784875.732601:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1412 00000100:00100000:2.0:1787784875.732612:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32157:x1874627658265344:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784875.732634:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784875.732654:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75e1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1714 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.732727:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc75e8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.732741:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc75e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.732753:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc75e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784875.732787:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2b:0x0]: returning size 0 00000004:00200000:2.0:1787784875.732792:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784875.732811:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc75e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.732836:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc75e1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1714 expref: 58 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.732852:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc75e8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1714 expref: 57 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.732860:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:2.0:1787784875.732863:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1787784875.732866:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac600, rc 0) 00000100:00100000:2.0:1787784875.732930:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+57:32157:x1874627658265344:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 320us (584us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784875.732938:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1412 00000100:00100000:1.0:1787784875.742889:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.745143:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658265472 00000100:00100000:3.0:1787784875.745190:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658265472 00000100:00100000:3.0:1787784875.745258:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 399 00000100:00100000:3.0:1787784875.745271:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658265472:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784875.745288:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784875.745308:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b171b expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784875.745331:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1787784875.745349:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=666s v=(0 0) 00010000:00010000:3.0:1787784875.745353:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784875.745362:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.745378:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.745398:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:3.0:1787784875.745450:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658265472:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 183us (2571us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.745457:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 399 00000100:00080000:0.0:1787784875.745584:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784875.754801:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.754884:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658265728 00000100:00100000:0.0:1787784875.754893:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658265728 00000100:00100000:0.0:1787784875.754969:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1413 00000100:00100000:0.0:1787784875.754980:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+57:32157:x1874627658265728:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784875.755094:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+57:32157:x1874627658265728:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 115us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784875.755102:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1413 00000100:00100000:0.0:1787784875.758992:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.759087:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658265856 00000100:00100000:2.0:1787784875.759096:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658265856 00000100:00100000:2.0:1787784875.759185:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:2.0:1787784875.759195:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:32157:x1874627658265856:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784875.759211:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784875.759229:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1722 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.759260:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.759270:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784875.759274:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784875.759279:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.759312:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.759326:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 12 pid: 13825 timeout: 766 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.759340:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668443648:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784875.759414:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895405dd9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784875.761616:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668443648:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784875.761652:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.761671:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac200, rc 0) 00000100:00100000:2.0:1787784875.761730:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:32157:x1874627658265856:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2536us (2756us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784875.761738:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 400 00000100:00100000:0.0:1787784875.763518:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.763649:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658265984 00000100:00100000:3.0:1787784875.763658:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658265984 00010000:00010000:3.0:1787784875.763694:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 12 pid: 13825 timeout: 766 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.763734:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 425 00000100:00100000:3.0:1787784875.763742:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658265984:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784875.763757:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784875.763776:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 12 pid: 13825 timeout: 766 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.763790:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 12 pid: 13825 timeout: 766 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.763811:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008481b064/0xb0ce63137acc75ef lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b171b expref: 12 pid: 13825 timeout: 766 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.763828:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784875.763832:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784875.763840:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.763850:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.763882:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.763894:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539379e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668443776:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784875.763994:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504c7ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784875.771752:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539379e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5796:1874627668443776:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784875.771783:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784875.771842:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658265984:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 8098us (8330us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.771853:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 425 00000100:00100000:1.0:1787784875.778777:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.778845:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658266112 00000100:00100000:3.0:1787784875.778855:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658266112 00002000:00100000:3.0:1787784875.778898:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1e:0x0] (0->157) req@ffff989403597c00 x1874627658266112/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:1/0 lens 648/0 e 0 to 0 dl 1787784886 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784875.778908:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403597c00 with x1874627658266112 ext(0->157) 00002000:00010000:3.0:1787784875.778923:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989403597c00 00000100:00100000:3.0:1787784875.778962:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 120 00000100:00100000:3.0:1787784875.778972:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403597c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658266112:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1787784875.779515:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403597c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658266112:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 543us (745us total) trans 4294967383 rc 0/0 00000100:00100000:3.0:1787784875.779528:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 120 00002000:00100000:3.0:1787784875.779535:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1e:0x0] (0->157) req@ffff989403597c00 x1874627658266112/t4294967383(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:1/0 lens 648/448 e 0 to 0 dl 1787784886 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784875.779544:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403597c00 with x1874627658266112 ext(0->157) 00002000:00010000:3.0:1787784875.779548:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989403597c00 00000100:00100000:0.0:1787784875.785555:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.785649:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658266240 00000100:00100000:2.0:1787784875.785659:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658266240 00000100:00100000:2.0:1787784875.785713:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1414 00000100:00100000:2.0:1787784875.785722:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:32157:x1874627658266240:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1787784875.853116:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968241 is committed 00000020:00080000:2.0:1787784875.853141:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 33 batchid updated 00000020:00080000:2.0:1787784875.853190:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 33 00000020:00080000:2.0:1787784875.853200:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 33 00000100:00100000:1.0:1787784875.854083:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32157:x1874627658266240:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 68353us (68531us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784875.854095:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1414 00000100:00100000:2.0:1787784875.876487:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.876683:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658266368 00000100:00100000:3.0:1787784875.876693:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658266368 00000100:00100000:3.0:1787784875.876754:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:3.0:1787784875.876764:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658266368:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000001:00080000:0.0:1787784875.915216:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967383 is committed 00000100:00100000:3.0:1787784875.915847:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658266368:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 38609us (38895us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.915859:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 401 00000100:00100000:2.0:1787784875.917758:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784875.918137:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658266624 00000100:00100000:1.0:1787784875.918245:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658266624 00000100:00100000:1.0:1787784875.918339:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1415 00000100:00100000:1.0:1787784875.918348:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32157:x1874627658266624:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1787784875.918393:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784875.918417:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc75e8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1714 expref: 51 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.918432:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:1.0:1787784875.918435:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784875.918459:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.918470:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.918476:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784875.918647:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d02ec3c9 reply state 000000001be9ee69 transno 4294968242 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.918654:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784875.918769:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32157:x1874627658266624:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 421us (1019us total) trans 4294968242 rc 0/0 00000100:00100000:1.0:1787784875.918774:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1415 00000100:00100000:0.0:1787784875.921276:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784875.921364:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.921377:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.921388:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.921394:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.921398:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.921405:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.921421:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:1.0:1787784875.921424:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784875.921428:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.921435:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc75fd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.921494:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658266752 00000100:00100000:3.0:1787784875.921502:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658266752 00000100:00100000:3.0:1787784875.921563:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1416 00000100:00100000:3.0:1787784875.921573:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32157:x1874627658266752:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784875.921596:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784875.921612:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1729 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.921665:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.921687:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.921711:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:3.0:1787784875.921713:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787784875.921716:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:3.0:1787784875.921776:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32157:x1874627658266752:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 204us (508us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.921784:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1416 00000100:00100000:2.0:1787784875.925723:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784875.925818:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658266880 00000100:00100000:1.0:1787784875.925828:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658266880 00000100:00100000:1.0:1787784875.925928:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 402 00000100:00100000:1.0:1787784875.925940:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658266880:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1787784875.926245:0:6670:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:1.0:1787784875.926250:0:6670:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb mtime from disk: 0 -> 1787784867 00002000:00010000:1.0:1787784875.926252:0:6670:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb atime from disk: 0 -> 1787784867 00002000:00010000:1.0:1787784875.926266:0:6670:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb ctime from disk: 0 -> 1787784867 00002000:00010000:1.0:1787784875.926269:0:6670:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1787784875.926332:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658266880:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 393us (617us total) trans 4294967384 rc 0/0 00000100:00100000:1.0:1787784875.926341:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 402 00000100:00100000:3.0:1787784875.936539:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784875.936890:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658267136 00000100:00100000:1.0:1787784875.936900:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658267136 00000100:00100000:1.0:1787784875.937063:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1417 00000100:00100000:1.0:1787784875.937075:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539379e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32157:x1874627658267136:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1787784875.937139:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.937177:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.937188:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.937225:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.937238:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 52 pid: 5812 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.937250:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668443904:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784875.937704:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1787784875.941435:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.941521:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658267264 00000100:00100000:2.0:1787784875.941529:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658267264 00010000:00010000:2.0:1787784875.941565:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 53 pid: 5812 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.941603:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 426 00000100:00100000:2.0:1787784875.941613:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2356:x1874627658267264:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784875.941628:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784875.941634:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 53 pid: 5812 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.941647:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 53 pid: 5812 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.941668:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:2.0:1787784875.941674:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.941682:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.941689:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.941712:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784875.941770:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2356:x1874627658267264:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 158us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784875.941777:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 426 00010000:00010000:1.0:1787784875.942648:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc75b7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ff expref: 52 pid: 5812 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.942668:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668443904:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784875.942692:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942701:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942707:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942736:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942744:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942750:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942755:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.942759:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942765:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942777:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:1.0:1787784875.942780:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787784875.942784:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.942790:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc760b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.942858:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539379e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32157:x1874627658267136:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 5785us (6327us total) trans 0 rc -22/-22 00000100:00100000:1.0:1787784875.942867:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1417 00010000:00010000:3.0:1787784875.953468:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784875.954891:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.955066:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658267392 00000100:00100000:2.0:1787784875.955075:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658267392 00000100:00100000:2.0:1787784875.955130:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 403 00000100:00100000:2.0:1787784875.955140:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658267392:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784875.955210:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.955232:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.955241:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1e:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784875.955245:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784875.955250:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.955433:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.955447:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 12 pid: 10688 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.955460:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668444032:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784875.955546:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787784875.960195:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668444032:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784875.960225:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784875.961802:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.962188:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658267520 00000100:00100000:2.0:1787784875.962198:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658267520 00010000:00010000:2.0:1787784875.962239:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 13 pid: 10688 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.962293:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 427 00000100:00100000:2.0:1787784875.962303:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658267520:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784875.962317:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784875.962337:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 13 pid: 10688 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.962351:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 13 pid: 10688 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.962374:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dd1d8d77/0xb0ce63137acc75f6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1722 expref: 13 pid: 10688 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.962392:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784875.962396:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784875.962401:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.962411:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.962451:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784875.962510:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658267520:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 208us (715us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784875.962519:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 427 00010000:00010000:3.0:1787784875.962820:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962831:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962855:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962864:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962872:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962878:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784875.962882:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962891:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962903:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.962910:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b4a8b6/0xb0ce63137acc7612 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.962977:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658267392:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 7838us (8092us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.962985:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 403 00000100:00100000:0.0:1787784875.968273:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784875.968381:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658267776 00000100:00100000:2.0:1787784875.968390:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658267776 00000100:00100000:2.0:1787784875.968447:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1418 00000100:00100000:2.0:1787784875.968457:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:32157:x1874627658267776:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784875.968546:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.968564:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.968576:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.968795:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.968807:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.968815:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.968848:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784875.968868:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 52 pid: 20438 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784875.968881:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f08780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668444160:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784875.969668:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cd4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784875.973306:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f08780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668444160:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784875.973337:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784875.975528:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784875.975741:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658267904 00000100:00100000:3.0:1787784875.975750:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658267904 00010000:00010000:3.0:1787784875.975791:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 53 pid: 20438 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784875.975829:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 428 00000100:00100000:3.0:1787784875.975839:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2356:x1874627658267904:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784875.975857:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784875.975863:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 53 pid: 20438 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.975876:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 53 pid: 20438 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.975902:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7604 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1729 expref: 53 pid: 20438 timeout: 767 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.975930:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:3.0:1787784875.975936:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.975959:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.975967:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784875.975993:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784875.976045:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:2356:x1874627658267904:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 206us (525us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784875.976054:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 428 00010000:00010000:1.0:1787784875.976440:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.976450:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784875.976633:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968243 00000001:00080000:1.0:1787784875.976805:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968244 00000001:00080000:1.0:1787784875.977186:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968245 00000100:00100000:1.0:1787784875.977272:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668444288:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787784875.977376:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784875.977401:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508cfe180 going to sleep for 11 seconds 00000100:00100000:0.0:1787784875.977475:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668444288 00000100:00100000:0.0:1787784875.977482:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668444288 00000100:00100000:0.0:1787784875.977534:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 71 00000100:00100000:0.0:1787784875.977542:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668444288:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784875.977616:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668444288:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784875.977625:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 71 00000100:00080000:1.0:1787784875.977746:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784875.977784:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668444288:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1787784875.978713:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968246 00000100:00100000:1.0:1787784875.978776:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668444416:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787784875.978816:0:10850:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784875.979091:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953efb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1787784875.980065:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668444416 00000100:00100000:0.0:1787784875.980074:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668444416 00000100:00100000:0.0:1787784875.980214:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 72 00000100:00100000:0.0:1787784875.980226:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:10850:x1874627668444416:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787784875.980487:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:10850:x1874627668444416:12345-0@lo:1000:mdt00_003.0 Request processed in 263us (1669us total) trans 4294967385 rc 0/0 00000100:00100000:0.0:1787784875.980496:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 72 00000100:00100000:1.0:1787784875.981856:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:10850:1874627668444416:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1787784875.982118:0:10850:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968247 00010000:00010000:1.0:1787784875.982192:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982203:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982210:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982216:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.982221:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982228:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982244:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff9895198b5e00) 00010000:00010000:1.0:1787784875.982247:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784875.982252:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982259:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7620 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982288:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982295:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982301:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982307:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.982309:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982315:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982324:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:1.0:1787784875.982326:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784875.982333:0:10850:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982344:0:10850:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982353:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.982388:0:10850:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 103 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.982400:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668444544:192.168.203.43@tcp:105: 00000100:00100000:1.0:1787784875.982525:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504c7ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784875.987334:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668444544:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784875.987366:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987382:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7555 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987413:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987427:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987436:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987447:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987453:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987459:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987465:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.987469:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987476:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987486:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:1.0:1787784875.987489:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784875.987492:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987499:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc7627 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987509:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987514:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987520:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987525:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784875.987527:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987543:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987552:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:1.0:1787784875.987555:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784875.987557:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784875.987563:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7619 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784875.987781:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32157:x1874627658267776:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 19323us (19514us total) trans 4294968248 rc 0/0 00000100:00100000:1.0:1787784875.987792:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1418 00000020:00080000:1.0:1787784875.987798:0:10850:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:0.0:1787784875.988551:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784875.988557:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784875.989048:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1787784875.989056:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784875.991759:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.992239:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658268032 00000100:00100000:0.0:1787784875.992250:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658268032 00000100:00100000:0.0:1787784875.992312:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:0.0:1787784875.992323:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d0850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32155:x1874627658268032:12345-192.168.203.43@tcp:49:cat.0 00000100:00100000:0.0:1787784875.992462:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d0850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32155:x1874627658268032:12345-192.168.203.43@tcp:49:cat.0 Request processed in 139us (711us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784875.992470:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 286 00000100:00100000:2.0:1787784875.997740:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784875.998065:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658268288 00000100:00100000:0.0:1787784875.998076:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658268288 00000100:00100000:0.0:1787784875.998132:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 287 00000100:00100000:0.0:1787784875.998142:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32157:x1874627658268288:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1787784875.999489:0:5815:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x2b:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1787784875.999677:0:5815:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968249 00010000:00010000:0.0:1787784875.999737:0:5815:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.999755:0:5815:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.999767:0:5815:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.999774:0:5815:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.999780:0:5815:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.999786:0:5815:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784875.999930:0:5815:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787784876.000046:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+58:32157:x1874627658268288:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1902us (2313us total) trans 4294968250 rc 0/0 00000100:00100000:0.0:1787784876.000056:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 287 00000020:00080000:0.0:1787784876.000061:0:5815:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1787784876.003374:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784876.003422:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784876.003516:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658268672 00000100:00100000:2.0:1787784876.003526:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658268672 00000100:00100000:2.0:1787784876.003600:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 65 00000100:00100000:2.0:1787784876.003611:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32154:x1874627658268672:12345-192.168.203.43@tcp:103:cat.0 00010000:00010000:2.0:1787784876.003628:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784876.003653:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc755c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b16ea expref: 104 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784876.003671:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff989531111b00) 00010000:00010000:2.0:1787784876.003680:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784876.003727:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:32154:x1874627658268672:12345-192.168.203.43@tcp:103:cat.0 Request processed in 118us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784876.003735:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 65 00000100:00100000:2.0:1787784876.005851:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784876.005935:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658268800 00000100:00100000:3.0:1787784876.005944:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658268800 00000100:00100000:3.0:1787784876.006005:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1419 00000100:00100000:3.0:1787784876.006014:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:32154:x1874627658268800:12345-192.168.203.43@tcp:58:cat.0 00000100:00100000:3.0:1787784876.006128:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:32154:x1874627658268800:12345-192.168.203.43@tcp:58:cat.0 Request processed in 115us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784876.006135:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1419 00000100:00100000:0.0:1787784876.006633:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658268416 00000100:00100000:0.0:1787784876.006639:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658268416 00000100:00100000:0.0:1787784876.006690:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00100000:0.0:1787784876.006699:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+15:32155:x1874627658268416:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784876.006711:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784876.006732:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc7635 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1730 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784876.006758:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1e:0x0] initial LVB size: 158, mtime: 0x6a8f6ea3, atime: 0x6a8f6ea3, ctime: 0x6a8f6ea3, blocks: 0xb 00010000:00010000:0.0:1787784876.006777:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=667s v=(0 0) 00010000:00010000:0.0:1787784876.006782:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784876.006791:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc7635 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1730 expref: 16 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784876.006807:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005bd332f0/0xb0ce63137acc7635 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1730 expref: 16 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784876.006818:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351400, rc 0) 00000100:00100000:0.0:1787784876.006848:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:32155:x1874627658268416:12345-192.168.203.43@tcp:101:cat.0 Request processed in 150us (3484us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784876.006858:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 404 00000100:00080000:2.0:1787784876.006930:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787784876.008521:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784876.008532:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.008552:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784876.008556:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.008565:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.008581:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784876.008584:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787784876.008589:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.008594:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc762e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5815 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.008618:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784876.008620:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784876.008622:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784876.012012:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784876.012090:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658268928 00000100:00100000:1.0:1787784876.012100:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658268928 00000100:00100000:1.0:1787784876.012186:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1420 00000100:00100000:1.0:1787784876.012196:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+103:32154:x1874627658268928:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784876.012216:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784876.012237:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc763c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1737 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.012302:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc763c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1737 expref: 104 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.012317:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641e00/0xb0ce63137acc763c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1737 expref: 104 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784876.012337:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff98951c407500) 00010000:00010000:1.0:1787784876.012340:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784876.012343:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641e00, rc 0) 00000100:00100000:1.0:1787784876.012406:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32154:x1874627658268928:12345-192.168.203.43@tcp:101:cat.0 Request processed in 211us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784876.012413:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1420 00000100:00100000:2.0:1787784876.016858:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784876.016925:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658269056 00000100:00100000:0.0:1787784876.016935:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658269056 00002000:00100000:0.0:1787784876.016990:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1e:0x0] (0->4095) req@ffff989508850780 x1874627658269056/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:2/0 lens 488/0 e 0 to 0 dl 1787784887 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784876.017002:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989508850780 with x1874627658269056 ext(0->4095) 00002000:00010000:0.0:1787784876.017007:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989508850780 00000100:00100000:0.0:1787784876.017034:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 121 00000100:00100000:0.0:1787784876.017042:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658269056:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:0.0:1787784876.017255:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658269056:12345-192.168.203.43@tcp:3:cat.0 Request processed in 213us (404us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784876.017263:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 121 00002000:00100000:0.0:1787784876.017270:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1e:0x0] (0->4095) req@ffff989508850780 x1874627658269056/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:2/0 lens 488/4536 e 0 to 0 dl 1787784887 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784876.017279:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989508850780 with x1874627658269056 ext(0->4095) 00002000:00010000:0.0:1787784876.017282:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989508850780 00000100:00100000:2.0:1787784876.025664:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784876.025840:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658269312 00000100:00100000:3.0:1787784876.025850:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658269312 00000100:00100000:3.0:1787784876.025903:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 288 00000100:00100000:3.0:1787784876.025924:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32155:x1874627658269312:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784876.026251:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32155:x1874627658269312:12345-192.168.203.43@tcp:35:cat.0 Request processed in 338us (596us total) trans 4294968251 rc 0/0 00000100:00100000:3.0:1787784876.026260:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 288 00000100:00100000:0.0:1787784876.529420:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668444672:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784876.529502:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784876.529681:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668444672 00000100:00100000:3.0:1787784876.529689:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668444672 00000100:00100000:3.0:1787784876.529747:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:3.0:1787784876.529757:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668444672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784876.529781:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3857711104 avail 3854565376 00000100:00100000:3.0:1787784876.529831:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668444672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (329us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784876.529838:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:3.0:1787784876.529883:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668444672:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784876.530130:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668444928:0@lo:400:kworker.0 00000100:00100000:3.0:1787784876.530182:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784876.530202:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784876.530208:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668445056:0@lo:400:kworker.0 00000100:00100000:3.0:1787784876.530230:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784876.530293:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668444928 00000100:00100000:3.0:1787784876.530297:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668444928 00000100:00100000:3.0:1787784876.530325:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445056 00000100:00100000:3.0:1787784876.530328:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445056 00000100:00100000:3.0:1787784876.530353:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:3.0:1787784876.530359:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668444928:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784876.530397:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668444928:12345-0@lo:400:kworker.0 Request processed in 37us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784876.530403:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:3.0:1787784876.530414:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1787784876.530421:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668445056:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784876.530444:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668445056:12345-0@lo:400:kworker.0 Request processed in 25us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784876.530449:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00080000:3.0:1787784876.530465:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784876.530495:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668445056:0@lo:400:kworker.0 00000100:00100000:3.0:1787784876.530505:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668444928:0@lo:400:kworker.0 00000100:00100000:0.0:1787784876.530630:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668445184:0@lo:400:kworker.0 00000100:00100000:0.0:1787784876.530659:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784876.530686:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1787784876.530692:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668444800:0@lo:400:kworker.0 00000100:00100000:0.0:1787784876.530718:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784876.530740:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668445312:0@lo:400:kworker.0 00000100:00100000:0.0:1787784876.530763:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784876.530785:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668444800 00000100:00100000:0.0:1787784876.530788:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668444800 00000100:00100000:0.0:1787784876.530815:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 608 00000100:00100000:0.0:1787784876.530820:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668444800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784876.530853:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668444800:12345-0@lo:400:kworker.0 Request processed in 32us (134us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784876.530859:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 608 00000100:00100000:0.0:1787784876.530893:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668444800:0@lo:400:kworker.0 00000100:00100000:1.0:1787784876.530965:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445312 00000100:00100000:1.0:1787784876.530969:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445312 00000100:00100000:1.0:1787784876.531011:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:1.0:1787784876.531018:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668445312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784876.531056:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668445312:12345-0@lo:400:kworker.0 Request processed in 39us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784876.531062:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:1.0:1787784876.531095:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98952175a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668445312:0@lo:400:kworker.0 00000100:00080000:0.0:1787784876.531143:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784876.531256:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445184 00000100:00100000:3.0:1787784876.531260:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445184 00000100:00100000:3.0:1787784876.531303:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1787784876.531310:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668445184:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784876.531347:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668445184:12345-0@lo:400:kworker.0 Request processed in 38us (687us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784876.531354:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1787784876.531390:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668445184:0@lo:400:kworker.0 00000100:00080000:0.0:1787784876.531441:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:3.0:1787784876.872552:0:22235:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784876.872562:0:22235:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784876.872608:0:22235:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 43 count 3 00000040:00080000:3.0:1787784876.872613:0:22235:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784876.987242:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784877.041789:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784877.041796:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784877.042610:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1787784877.042621:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784877.836410:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784877.836471:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784877.836509:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784877.836548:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784877.836570:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658269696 00000100:00100000:3.0:1787784877.836584:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784877.836587:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658269696 00000100:00100000:3.0:1787784877.836629:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784877.836674:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784877.836734:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658269952 00000100:00100000:1.0:1787784877.836742:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658269952 00000100:00100000:2.0:1787784877.836754:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658269440 00000100:00100000:2.0:1787784877.836765:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658269440 00000100:00100000:1.0:1787784877.836825:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658269568 00000100:00100000:2.0:1787784877.836834:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 609 00000100:00100000:1.0:1787784877.836835:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658269568 00000100:00100000:2.0:1787784877.836851:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658269440:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784877.836925:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 479 00000100:00100000:1.0:1787784877.836955:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+58:2355:x1874627658269952:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784877.836966:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658269440:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 118us (569us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784877.836976:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 609 00000100:00100000:1.0:1787784877.837243:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+58:2355:x1874627658269952:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 212us (637us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784877.837252:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 479 00000100:00100000:1.0:1787784877.837281:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 480 00000100:00100000:1.0:1787784877.837288:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:2355:x1874627658269568:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784877.837368:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:2355:x1874627658269568:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 70us (776us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784877.837375:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 480 00000100:00080000:1.0:1787784877.837446:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787784877.837461:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784877.840082:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270208 00000100:00100000:0.0:1787784877.840096:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270208 00000100:00100000:0.0:1787784877.840184:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658269824 00000100:00100000:0.0:1787784877.840188:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658269824 00000100:00100000:0.0:1787784877.840204:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270080 00000100:00100000:0.0:1787784877.840206:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270080 00000100:00100000:0.0:1787784877.840239:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:0.0:1787784877.840250:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658269696:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784877.840317:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658269696:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 70us (3845us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784877.840324:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:0.0:1787784877.840343:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 446 00000100:00100000:0.0:1787784877.840901:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658270208:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784877.840965:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658270208:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 70us (4418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784877.841023:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00100000:3.0:1787784877.841052:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658269824:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784877.841186:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 448 00000100:00100000:2.0:1787784877.841200:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658270080:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784877.841255:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2355:x1874627658270080:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (4581us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784877.841264:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 448 00000100:00100000:3.0:1787784877.841453:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658269824:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 416us (4824us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784877.841462:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00080000:2.0:1787784877.842336:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784877.842462:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784877.848794:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 446 00010000:00010000:3.0:1787784878.003524:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784878.674686:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967373 is committed 00010000:00010000:1.0:1787784879.031867:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784879.602761:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668445440:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784879.602836:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784879.602957:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445440 00000100:00100000:2.0:1787784879.602966:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445440 00000100:00100000:2.0:1787784879.603075:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1787784879.603086:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536d3cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668445440:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784879.603122:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3855613952 00000100:00080000:0.0:1787784879.603169:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784879.603451:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536d3cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668445440:12345-0@lo:13:osp-pre-0-0.0 Request processed in 366us (613us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784879.603453:0:3278:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967373 gen 2 00000100:00100000:2.0:1787784879.603463:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000004:00080000:3.0:1787784879.603589:0:3278:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989540666d00, transno 4294967373 00000100:00100000:3.0:1787784879.603655:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668445440:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787784880.049372:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784881.074729:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784881.183230:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967385 is committed 00000001:00080000:3.0:1787784881.183249:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967384 is committed 00000001:00080000:0.0:1787784881.196298:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968250 is committed 00000020:00080000:0.0:1787784881.196336:0:5833:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff98950310dc00 refcount 2 committed 0 result 0 batchid 34 00000020:00080000:0.0:1787784881.196343:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989401288b00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff98950310d540 00000020:00080000:0.0:1787784881.196346:0:5833:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff989401288400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9895393b1800 00000001:00080000:0.0:1787784881.196367:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968251 is committed 00000020:00080000:0.0:1787784881.196460:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 34 committed batchid 33 00000020:00080000:0.0:1787784881.196890:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 33 00000100:00100000:3.0:1787784881.649538:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668445952:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.649607:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784881.649674:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1787784881.649680:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668445568:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.649712:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784881.649734:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668446080:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.649762:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784881.649792:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668445696:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.649816:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784881.649835:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668446208:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.649859:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784881.649875:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784881.649881:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668445824:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.649904:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784881.649968:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445952 00000100:00100000:3.0:1787784881.649976:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445952 00000100:00100000:3.0:1787784881.650028:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668446208 00000100:00100000:3.0:1787784881.650032:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668446208 00000100:00100000:3.0:1787784881.650055:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:3.0:1787784881.650063:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668445952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.650115:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668445952:12345-0@lo:400:kworker.0 Request processed in 51us (507us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784881.650122:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:3.0:1787784881.650138:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1787784881.650144:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668446208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.650206:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668446208:12345-0@lo:400:kworker.0 Request processed in 62us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784881.650212:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1787784881.650431:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445568 00000100:00100000:3.0:1787784881.650435:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445568 00000100:00100000:3.0:1787784881.650508:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:3.0:1787784881.650515:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668445568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.650552:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668445568:12345-0@lo:400:kworker.0 Request processed in 38us (840us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784881.650558:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:3.0:1787784881.654108:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668446080 00000100:00100000:3.0:1787784881.654123:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668446080 00000100:00100000:3.0:1787784881.654202:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445696 00000100:00100000:3.0:1787784881.654206:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445696 00000100:00100000:3.0:1787784881.654225:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668445824 00000100:00100000:3.0:1787784881.654228:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668445824 00000100:00080000:0.0:1787784881.654263:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784881.654268:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784881.654271:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784881.654289:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1787784881.654300:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668446080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784881.654339:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 482 00000100:00100000:2.0:1787784881.654349:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668445696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784881.654385:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668446080:12345-0@lo:400:kworker.0 Request processed in 87us (4620us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784881.654392:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:2.0:1787784881.654459:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668445696:12345-0@lo:400:kworker.0 Request processed in 111us (4641us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784881.654838:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 482 00000100:00100000:3.0:1787784881.654959:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:3.0:1787784881.654977:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668445824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.655014:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668445824:12345-0@lo:400:kworker.0 Request processed in 48us (5109us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784881.655020:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:3.0:1787784881.655485:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b534b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668446336:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784881.655745:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668445568:0@lo:400:kworker.0 00000100:00100000:2.0:1787784881.655761:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668446080:0@lo:400:kworker.0 00000100:00100000:2.0:1787784881.655772:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668445952:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.655947:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784881.656003:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668446336 00000100:00100000:3.0:1787784881.656009:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668446336 00000100:00100000:3.0:1787784881.656042:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1787784881.656050:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668446336:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784881.656081:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784881.656122:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668446336:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784881.656128:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1787784881.656176:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668445824:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.656189:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668445696:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.656197:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967385 gen 2 00000004:00080000:3.0:1787784881.656202:0:3276:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967385 committed transno 4294967385 00000100:00100000:3.0:1787784881.656220:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668446208:0@lo:400:kworker.0 00000100:00100000:3.0:1787784881.656264:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b534b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668446336:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1787784882.109574:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784882.162613:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784882.162622:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784882.162918:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 43 count 2 00000040:00080000:3.0:1787784882.162927:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784882.381086:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784882.381203:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270336 00000100:00100000:0.0:1787784882.381212:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270336 00000100:00100000:0.0:1787784882.381283:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1421 00000100:00100000:0.0:1787784882.381295:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32127:x1874627658270336:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1787784882.381393:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32127:x1874627658270336:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 101us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784882.381405:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1421 00000100:00080000:2.0:1787784882.383280:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784882.952449:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784882.952497:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784882.952531:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784882.952572:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784882.952597:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784882.952614:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784882.952652:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784882.952677:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270720 00000100:00100000:2.0:1787784882.952686:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270720 00000100:00100000:2.0:1787784882.952736:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658271232 00000100:00100000:2.0:1787784882.952739:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658271232 00000100:00100000:2.0:1787784882.952753:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270848 00000100:00100000:2.0:1787784882.952755:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270848 00000100:00100000:2.0:1787784882.952765:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658271104 00000100:00100000:2.0:1787784882.952768:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658271104 00000100:00100000:2.0:1787784882.952784:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:2.0:1787784882.952792:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658270720:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784882.952858:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658270720:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 66us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784882.952865:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:2.0:1787784882.952881:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:2.0:1787784882.952887:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658271232:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784882.952926:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2354:x1874627658271232:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784882.952932:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:2.0:1787784882.952944:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 453 00000100:00100000:2.0:1787784882.952950:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658270848:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784882.952970:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658270848:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784882.952976:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 453 00000100:00100000:2.0:1787784882.952985:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 454 00000100:00100000:2.0:1787784882.952991:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658271104:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784882.953009:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2354:x1874627658271104:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784882.953020:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 454 00000100:00080000:2.0:1787784882.953035:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784882.953039:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784882.953290:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270464 00000100:00100000:0.0:1787784882.953295:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270464 00000100:00100000:0.0:1787784882.953332:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 611 00000100:00100000:0.0:1787784882.953339:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658270464:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784882.953370:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658270464:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (931us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784882.953376:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 611 00000100:00100000:1.0:1787784882.953452:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270976 00000100:00100000:1.0:1787784882.953457:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270976 00000100:00100000:1.0:1787784882.953494:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658270592 00000100:00100000:1.0:1787784882.953497:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658270592 00000100:00100000:1.0:1787784882.953522:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 484 00000100:00100000:1.0:1787784882.953529:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2354:x1874627658270976:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784882.953558:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2354:x1874627658270976:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (1027us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784882.953565:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 484 00000100:00100000:1.0:1787784882.953575:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00100000:1.0:1787784882.953581:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:2354:x1874627658270592:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784882.953601:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:2354:x1874627658270592:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (1004us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784882.953607:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00080000:1.0:1787784882.953632:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784883.121440:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784884.146517:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784884.734848:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784884.734876:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668446464:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784884.734975:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784884.737735:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668446464 00000100:00100000:3.0:1787784884.737753:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668446464 00000100:00100000:3.0:1787784884.737840:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1787784884.737851:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403597480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668446464:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784884.737909:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784884.738007:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403597480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668446464:12345-0@lo:13:osp-pre-0-0.0 Request processed in 158us (3032us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784884.738015:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00080000:1.0:1787784884.738655:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784884.738719:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668446464:0@lo:13:osp-pre-0-0.0 00000001:02000400:3.0:1787784884.812710:0:22438:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 18:54:42 (1787784882) 00000100:00100000:2.0:1787784885.152650:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.152762:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658271360 00000100:00100000:3.0:1787784885.152775:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658271360 00000100:00100000:3.0:1787784885.152861:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1422 00000100:00100000:3.0:1787784885.152874:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32770:x1874627658271360:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1787784885.153021:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32770:x1874627658271360:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 149us (384us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784885.153031:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1422 00000100:00080000:2.0:1787784885.153531:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784885.174604:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784885.186418:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784885.186465:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658271488 00000100:00100000:2.0:1787784885.186477:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658271488 00000100:00100000:2.0:1787784885.186554:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1423 00000100:00100000:2.0:1787784885.186566:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32771:x1874627658271488:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784885.186592:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784885.186878:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7643 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b173e expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.186992:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187007:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187019:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187062:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187070:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187079:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187534:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187544:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187550:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187557:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784885.187562:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187602:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187637:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:2.0:1787784885.187641:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784885.187648:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187655:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc764a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187676:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187682:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187688:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187693:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784885.187819:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187826:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187834:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff989520131500) 00010000:00010000:2.0:1787784885.187837:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784885.187840:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187850:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7651 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187898:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7643 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b173e expref: 105 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187909:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:2.0:1787784885.187911:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784885.187920:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7643 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b173e expref: 105 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.187929:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad000, rc 0) 00000100:00100000:2.0:1787784885.188021:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32771:x1874627658271488:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1456us (1614us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784885.188030:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1423 00000100:00080000:2.0:1787784885.188078:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784885.199046:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784885.199193:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658271744 00000100:00100000:2.0:1787784885.199206:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658271744 00000100:00100000:2.0:1787784885.199265:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1424 00000100:00100000:2.0:1787784885.199275:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32771:x1874627658271744:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:2.0:1787784885.199350:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784885.199384:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc738e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1649 expref: 104 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.199403:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:2.0:1787784885.199411:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784885.199740:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.199757:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.199767:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.199805:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 48 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.199823:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 48 pid: 5811 timeout: 776 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784885.199839:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031261c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668446592:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784885.199959:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784885.205447:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031261c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668446592:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784885.205491:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784885.206962:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.207143:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658271872 00000100:00100000:3.0:1787784885.207181:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658271872 00010000:00010000:3.0:1787784885.207232:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 49 pid: 5811 timeout: 776 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784885.207276:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 429 00000100:00100000:3.0:1787784885.207288:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658271872:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784885.207307:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784885.207316:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 49 pid: 5811 timeout: 776 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.207331:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 49 pid: 5811 timeout: 776 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.207361:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc73cd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b165e expref: 49 pid: 5811 timeout: 776 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.207379:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:3.0:1787784885.207387:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.207399:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.207408:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.207440:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784885.207512:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+48:2357:x1874627658271872:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 226us (561us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784885.207522:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 429 00010000:00010000:1.0:1787784885.208267:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.208279:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.208314:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.208326:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.208335:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784885.209750:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 512 00010000:00010000:1.0:1787784885.209768:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.209784:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.209792:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.209798:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784885.209803:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.209812:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.209828:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9895198b5a00) 00010000:00010000:1.0:1787784885.209831:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784885.209837:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.209843:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7658 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784885.209864:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000013943eba reply state 000000007d87d40b transno 4294968252 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784885.209873:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787784885.210104:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32771:x1874627658271744:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 10825us (11063us total) trans 4294968252 rc 0/0 00000100:00100000:1.0:1787784885.210114:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1424 00010000:00010000:3.0:1787784885.212360:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.212379:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.212386:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.212391:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.212396:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.212405:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.212422:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff98951c407700) 00010000:00010000:3.0:1787784885.212425:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.212430:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.212436:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc765f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784885.219682:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784885.220355:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272000 00000100:00100000:1.0:1787784885.220369:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272000 00000100:00100000:1.0:1787784885.220448:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1425 00000100:00100000:1.0:1787784885.220461:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+104:32772:x1874627658272000:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784885.220502:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784885.220535:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7666 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1745 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.220622:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc766d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.220640:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc766d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.220654:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc766d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784885.220735:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784885.220740:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 34 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784885.220772:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc766d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.220803:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc7666 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1745 expref: 106 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.220827:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc766d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1745 expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.220838:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784885.220841:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784885.220845:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641400, rc 0) 00000100:00100000:1.0:1787784885.221019:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32772:x1874627658272000:12345-192.168.203.43@tcp:101:bash.0 Request processed in 560us (1361us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784885.221028:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1425 00000100:00100000:2.0:1787784885.224618:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784885.224773:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272128 00000100:00100000:0.0:1787784885.224786:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272128 00000100:00100000:0.0:1787784885.224899:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1426 00000100:00100000:0.0:1787784885.224912:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+105:32772:x1874627658272128:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784885.224939:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784885.224966:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7674 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b174c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.225050:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.225067:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.225081:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.225100:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.225108:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.225116:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228058:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7689 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228083:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7689 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228096:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7689 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784885.228200:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 512 00000004:00200000:0.0:1787784885.228204:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784885.228233:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7689 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228247:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228254:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228260:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228266:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784885.228269:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228277:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228291:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784885.228294:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784885.228298:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228304:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc767b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228319:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228325:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228330:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228334:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784885.228337:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228343:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228349:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff989520108600) 00010000:00010000:0.0:1787784885.228352:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784885.228354:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228364:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7682 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228407:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc7674 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b174c expref: 107 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228420:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150be00/0xb0ce63137acc7689 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b174c expref: 106 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.228427:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784885.228429:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784885.228431:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150be00, rc 0) 00000100:00100000:0.0:1787784885.228509:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32772:x1874627658272128:12345-192.168.203.43@tcp:101:bash.0 Request processed in 3599us (3904us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784885.228518:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1426 00000040:00080000:2.0:1787784885.233851:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784885.233862:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784885.234730:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:2.0:1787784885.234742:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784885.235913:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.236525:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272256 00000100:00100000:3.0:1787784885.236541:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272256 00000100:00100000:3.0:1787784885.236619:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1427 00000100:00100000:3.0:1787784885.236630:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32772:x1874627658272256:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784885.236657:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784885.236683:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7690 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1753 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.236762:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.236778:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.236791:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.236923:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.236944:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.236956:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237042:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237055:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237062:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237069:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.237074:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237081:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237093:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.237096:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.237358:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237367:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7697 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237382:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237389:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237396:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237402:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.237406:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237414:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237426:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5000) 00010000:00010000:3.0:1787784885.237429:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.237432:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237437:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc769e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237488:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7690 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1753 expref: 107 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237499:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.237502:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.237506:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7690 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1753 expref: 107 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.237515:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784885.237578:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32772:x1874627658272256:12345-192.168.203.43@tcp:101:bash.0 Request processed in 950us (1673us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784885.237587:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1427 00000100:00100000:3.0:1787784885.247415:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784885.247521:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272384 00000100:00100000:1.0:1787784885.247534:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272384 00000100:00100000:1.0:1787784885.247615:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1428 00000100:00100000:1.0:1787784885.247626:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393761c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+48:32773:x1874627658272384:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784885.247654:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784885.247692:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc76a5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b175a expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.247777:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.247794:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.247807:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784885.247899:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:1.0:1787784885.247904:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 34 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784885.247933:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.247965:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc76a5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.247987:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.248003:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784885.248006:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784885.248010:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641800, rc 0) 00000100:00100000:1.0:1787784885.248088:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393761c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32773:x1874627658272384:12345-192.168.203.43@tcp:101:rm.0 Request processed in 462us (687us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784885.248096:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1428 00000100:00100000:2.0:1787784885.253995:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784885.254089:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272512 00000100:00100000:0.0:1787784885.254105:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272512 00000100:00100000:0.0:1787784885.254232:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1429 00000100:00100000:0.0:1787784885.254244:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+49:32773:x1874627658272512:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784885.254270:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784885.254306:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc76b3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1761 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255008:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255030:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255043:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255058:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255065:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255071:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255296:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc76c8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255306:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc76c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255363:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc76c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784885.255403:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 512 00000004:00200000:0.0:1787784885.255407:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784885.255432:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc76c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255446:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255457:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255464:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255594:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784885.255599:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255608:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255621:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784885.255625:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784885.255630:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255637:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc76ba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255652:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255659:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255665:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255669:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784885.255673:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255679:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255687:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff989520108600) 00010000:00010000:0.0:1787784885.255689:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784885.255692:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255698:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc76c1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255738:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc76b3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1761 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255753:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403350200/0xb0ce63137acc76c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1761 expref: 50 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.255762:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784885.255764:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784885.255767:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403350200, rc 0) 00000100:00100000:0.0:1787784885.255852:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32773:x1874627658272512:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1609us (1865us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784885.255860:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1429 00000100:00100000:0.0:1787784885.261222:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.261450:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272640 00000100:00100000:3.0:1787784885.261463:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272640 00000100:00100000:3.0:1787784885.261711:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1430 00000100:00100000:3.0:1787784885.261723:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32773:x1874627658272640:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784885.261751:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784885.261779:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc76cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1768 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.261869:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.261896:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.261910:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.261927:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.261933:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.261939:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262021:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262029:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262036:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262043:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.262047:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262061:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262076:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.262079:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.262084:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262091:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc76d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262107:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262113:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262120:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262125:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.262128:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262134:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262141:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5000) 00010000:00010000:3.0:1787784885.262144:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.262178:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262189:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc76dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262229:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc76cf lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1768 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262240:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.262242:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.262246:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc76cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1768 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.262255:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784885.262448:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32773:x1874627658272640:12345-192.168.203.43@tcp:101:rm.0 Request processed in 724us (1239us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784885.262455:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1430 00000100:00100000:2.0:1787784885.268986:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.269081:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272768 00000100:00100000:3.0:1787784885.269093:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272768 00000100:00100000:3.0:1787784885.269481:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1431 00000100:00100000:3.0:1787784885.269496:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32773:x1874627658272768:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784885.269524:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784885.269551:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc76e4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b176f expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269648:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269666:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269694:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269713:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269721:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269728:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269813:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269822:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269829:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269836:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.269841:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269849:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269863:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.269867:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.269872:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269880:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc76eb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269908:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269915:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269921:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269926:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.269929:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269934:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269942:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5f00) 00010000:00010000:3.0:1787784885.269944:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.269947:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269954:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc76f2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.269994:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc76e4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b176f expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.270012:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.270015:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.270018:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc76e4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b176f expref: 51 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.270029:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784885.270204:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32773:x1874627658272768:12345-192.168.203.43@tcp:101:rm.0 Request processed in 711us (1232us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784885.270213:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1431 00000100:00100000:2.0:1787784885.291682:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784885.293101:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658272896 00000100:00100000:0.0:1787784885.293120:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658272896 00000100:00100000:0.0:1787784885.293255:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1432 00000100:00100000:0.0:1787784885.293269:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32774:x1874627658272896:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784885.293296:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784885.293327:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc76f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1776 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293418:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293434:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293447:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293465:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293473:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293479:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293563:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293571:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293579:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293589:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784885.293594:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293603:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293618:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784885.293621:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784885.293627:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293634:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc7700 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293650:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293657:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293663:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293668:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784885.293671:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293678:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293689:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520108600) 00010000:00010000:0.0:1787784885.293692:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784885.293695:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293702:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7707 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293748:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc76f9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1776 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293765:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784885.293768:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784885.293772:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc76f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1776 expref: 51 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.293782:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:0.0:1787784885.293869:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32774:x1874627658272896:12345-192.168.203.43@tcp:101:cp.0 Request processed in 603us (2205us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784885.293878:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1432 00000100:00100000:0.0:1787784885.307114:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.307224:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658273024 00000100:00100000:3.0:1787784885.307240:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658273024 00000100:00100000:3.0:1787784885.307306:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1433 00000100:00100000:3.0:1787784885.307317:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32774:x1874627658273024:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784885.307338:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784885.307361:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc770e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b177d expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307432:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307447:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307458:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307486:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307494:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307501:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307566:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307574:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307581:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307587:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.307592:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307601:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307613:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.307617:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.307621:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307628:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7715 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307642:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307649:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307655:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307660:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.307664:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307681:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307690:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5f00) 00010000:00010000:3.0:1787784885.307693:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.307697:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307703:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc771c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307737:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc770e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b177d expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307748:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.307751:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.307754:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc770e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b177d expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.307769:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784885.307846:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32774:x1874627658273024:12345-192.168.203.43@tcp:101:cp.0 Request processed in 531us (746us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784885.307854:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1433 00000100:00100000:2.0:1787784885.325741:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.326231:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658273280 00000100:00100000:3.0:1787784885.326241:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658273280 00000100:00100000:3.0:1787784885.326302:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1434 00000100:00100000:3.0:1787784885.326313:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32774:x1874627658273280:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784885.326333:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784885.326353:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7723 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1784 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.326485:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.326500:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.326512:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.326527:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.326534:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.326540:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787784885.327629:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1c:0x0], index 0: 0 00010000:00010000:3.0:1787784885.327815:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.327826:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.327835:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.327867:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.327876:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.329196:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.329205:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784885.329209:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.329218:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.329234:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784885.329237:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784885.329242:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.329255:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc772a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784885.329274:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000026a87baf reply state 00000000d656bfdd transno 4294968253 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784885.329283:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784885.329482:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7723 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 53 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.329501:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 52 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.329509:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784885.329512:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784885.329514:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:2.0:1787784885.329594:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32774:x1874627658273280:12345-192.168.203.43@tcp:101:cp.0 Request processed in 3282us (3861us total) trans 4294968253 rc 0/0 00000100:00100000:2.0:1787784885.329603:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1434 00010000:00010000:1.0:1787784885.333386:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.333404:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.333412:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.333419:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784885.333425:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.333433:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.333449:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5f00) 00010000:00010000:1.0:1787784885.333452:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784885.333456:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.333462:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7731 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784885.347033:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784885.347314:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658273408 00000100:00100000:0.0:1787784885.347323:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658273408 00000100:00100000:0.0:1787784885.347386:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:0.0:1787784885.347396:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658273408:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784885.347419:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784885.347437:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b178b expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784885.347463:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784885.347481:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1787784885.347486:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784885.347496:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.347512:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 9 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784885.347525:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00080000:2.0:1787784885.347638:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784885.347785:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658273408:12345-192.168.203.43@tcp:101:cp.0 Request processed in 389us (759us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784885.347793:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 405 00000100:00100000:0.0:1787784885.357543:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.357728:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658273536 00000100:00100000:3.0:1787784885.357738:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658273536 00000100:00100000:3.0:1787784885.357805:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:3.0:1787784885.357817:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:32774:x1874627658273536:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784885.357834:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784885.357853:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1792 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.357901:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.357911:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784885.357916:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784885.357921:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.357955:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 10 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.358086:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 10 pid: 6670 timeout: 776 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784885.358105:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668446720:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784885.358227:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051db40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784885.362504:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668446720:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784885.362542:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 10 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.362562:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784885.362620:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:32774:x1874627658273536:12345-192.168.203.43@tcp:101:cp.0 Request processed in 4805us (5087us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784885.362628:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 406 00000100:00100000:2.0:1787784885.370254:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.370349:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658273664 00000100:00100000:3.0:1787784885.370360:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658273664 00010000:00010000:3.0:1787784885.370399:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 10 pid: 6670 timeout: 776 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784885.370441:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:3.0:1787784885.370452:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c94c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658273664:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784885.370479:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784885.370502:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 10 pid: 6670 timeout: 776 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.370516:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 10 pid: 6670 timeout: 776 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.370544:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eced4e3f/0xb0ce63137acc7746 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b178b expref: 10 pid: 6670 timeout: 776 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.370560:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784885.370564:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784885.370572:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.370580:0:21579:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.370615:0:21579:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784885.370627:0:21579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:21579:1874627668446848:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784885.370708:0:21579:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051db40 going to sleep for 11 seconds 00000100:00100000:0.0:1787784885.374771:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784885.374849:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658273920 00000100:00100000:2.0:1787784885.374857:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658273920 00000100:00100000:2.0:1787784885.374940:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 289 00000100:00100000:2.0:1787784885.374949:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32774:x1874627658273920:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:2.0:1787784885.375285:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32774:x1874627658273920:12345-192.168.203.43@tcp:35:cp.0 Request processed in 336us (521us total) trans 4294968254 rc 0/0 00000100:00100000:2.0:1787784885.375293:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 289 00000100:00100000:3.0:1787784885.375678:0:21579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:21579:1874627668446848:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784885.375699:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784885.375721:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c94c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658273664:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5271us (5478us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784885.375730:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:0.0:1787784885.445213:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784885.445330:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658274048 00000100:00100000:1.0:1787784885.445350:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658274048 00000100:00100000:1.0:1787784885.445414:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1435 00000100:00100000:1.0:1787784885.445425:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ceed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32777:x1874627658274048:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784885.445447:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784885.445481:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7754 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1799 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445557:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445572:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445584:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445600:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445608:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445615:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445713:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7769 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445723:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7769 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445731:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7769 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784885.445764:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2c:0x0]: returning size 158 00000004:00200000:1.0:1787784885.445768:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784885.445786:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7769 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445798:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7769 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445807:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445813:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445820:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445827:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784885.445831:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445840:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445852:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784885.445855:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784885.445861:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445872:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc775b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445900:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445906:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445913:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445918:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784885.445922:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445928:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445935:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951c407700) 00010000:00010000:1.0:1787784885.445938:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784885.445941:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445948:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc7762 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445979:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7754 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1799 expref: 108 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.445998:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7769 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1799 expref: 107 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.446005:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:1.0:1787784885.446008:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784885.446010:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3a00, rc 0) 00000100:00100000:1.0:1787784885.446653:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ceed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32777:x1874627658274048:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1227us (1448us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784885.446663:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1435 00000100:00100000:1.0:1787784885.456900:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784885.457018:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658274176 00000100:00100000:3.0:1787784885.457026:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658274176 00000100:00100000:3.0:1787784885.457093:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 407 00000100:00100000:3.0:1787784885.457103:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658274176:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784885.457120:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784885.457136:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc7770 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17a0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784885.457215:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668446976:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784885.457297:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d540 going to sleep for 11 seconds 00000100:00080000:2.0:1787784885.459128:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1787784885.464940:0:6669:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1787784885.464949:0:6669:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb mtime: 0 -> 1787784884 00002000:00010000:3.0:1787784885.464951:0:6669:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb atime: 0 -> 1787784884 00002000:00010000:3.0:1787784885.464954:0:6669:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb ctime: 0 -> 1787784884 00000100:00100000:3.0:1787784885.464974:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063361c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668446976:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784885.465008:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc7770 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17a0 expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.465030:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc7770 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17a0 expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784885.465043:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:3.0:1787784885.465097:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658274176:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7995us (8220us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784885.465106:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 407 00000100:00100000:1.0:1787784885.468552:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784885.469022:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658274304 00000100:00100000:2.0:1787784885.469033:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658274304 00000100:00100000:2.0:1787784885.469142:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1436 00000100:00100000:2.0:1787784885.469187:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32777:x1874627658274304:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784885.469209:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784885.469230:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7777 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17a7 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.469310:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc777e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.469325:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc777e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.469337:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc777e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784885.469393:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 512 00000004:00200000:2.0:1787784885.469397:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784885.469420:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc777e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.469445:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7777 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17a7 expref: 109 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.469462:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc777e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17a7 expref: 108 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784885.469484:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784885.469487:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784885.469491:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad000, rc 0) 00000100:00100000:2.0:1787784885.469554:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32777:x1874627658274304:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 371us (1012us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784885.469562:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1436 00000100:00100000:3.0:1787784885.473253:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784885.473845:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658274432 00000100:00100000:1.0:1787784885.473855:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658274432 00000100:00100000:1.0:1787784885.473932:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 408 00000100:00100000:1.0:1787784885.473942:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658274432:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784885.473961:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784885.473980:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b283eea7/0xb0ce63137acc7785 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17ae expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784885.474038:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668447104:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784885.474117:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784885.477562:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668447104:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784885.477604:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b283eea7/0xb0ce63137acc7785 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ae expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.477995:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b283eea7/0xb0ce63137acc7785 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ae expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784885.478011:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3400, rc 0) 00000100:00100000:1.0:1787784885.478066:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658274432:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4125us (4834us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784885.478075:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 408 00000100:00100000:1.0:1787784885.484956:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784885.485024:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658274688 00000100:00100000:2.0:1787784885.485034:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658274688 00000100:00100000:2.0:1787784885.485087:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1437 00000100:00100000:2.0:1787784885.485097:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32777:x1874627658274688:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:2.0:1787784885.485365:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].43, 136 off14040 00000100:00100000:2.0:1787784885.485438:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32777:x1874627658274688:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 342us (489us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784885.485445:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1437 00010000:00010000:2.0:1787784886.194606:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784886.257529:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784886.257543:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784886.257592:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 44 count 3 00000040:00080000:2.0:1787784886.257601:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784886.257969:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035da580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668447232:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784886.258118:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f900 going to sleep for 11 seconds 00000100:00100000:2.0:1787784886.264358:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035da580 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668447232:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784886.279629:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784886.279731:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658274816 00000100:00100000:1.0:1787784886.279743:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658274816 00000100:00100000:1.0:1787784886.279809:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1438 00000100:00100000:1.0:1787784886.279820:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658274816:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784886.279865:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784886.280020:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658274816:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 201us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784886.280028:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1438 00000100:00100000:2.0:1787784886.300768:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784886.300814:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658275072 00000100:00100000:2.0:1787784886.300825:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658275072 00000100:00100000:2.0:1787784886.300903:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1439 00000100:00100000:2.0:1787784886.300916:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658275072:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784886.300943:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784886.300968:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc778c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17b5 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.301558:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7793 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.301583:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7793 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.301678:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7793 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784886.301694:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2c:0x0]: returning size 158 00010000:00010000:2.0:1787784886.302172:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc778c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17b5 expref: 56 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.302199:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7793 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17b5 expref: 55 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.302210:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784886.302213:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784886.302217:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac200, rc 0) 00000100:00100000:2.0:1787784886.304828:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32787:x1874627658275072:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 3897us (4056us total) trans 4294968255 rc 0/0 00000100:00100000:2.0:1787784886.304843:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1439 00000100:00100000:0.0:1787784886.320697:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784886.320735:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.320987:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658275200 00000100:00100000:3.0:1787784886.321000:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658275200 00000100:00100000:3.0:1787784886.321069:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 409 00000100:00100000:3.0:1787784886.321081:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658275200:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784886.321136:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.321185:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.321195:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1c:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784886.321200:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784886.321205:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.321472:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 9 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.321490:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 9 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784886.321507:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668447360:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784886.321614:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d780 going to sleep for 11 seconds 00000100:00100000:2.0:1787784886.322170:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658275328 00000100:00100000:2.0:1787784886.322178:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658275328 00000100:00100000:2.0:1787784886.322228:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 66 00000100:00100000:2.0:1787784886.322246:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ced2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+55:2357:x1874627658275328:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784886.322261:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784886.322298:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7793 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17b5 expref: 55 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.322322:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784886.322328:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784886.322373:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ced2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658275328:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 129us (1637us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784886.322382:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 66 00000100:00100000:2.0:1787784886.342399:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895063370c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668447360:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784886.342443:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784886.344507:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.344572:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658275456 00000100:00100000:3.0:1787784886.344581:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658275456 00002000:00100000:3.0:1787784886.344634:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1c:0x0] (0->157) req@ffff9895063370c0 x1874627658275456/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:12/0 lens 648/0 e 0 to 0 dl 1787784897 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784886.344647:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895063370c0 with x1874627658275456 ext(0->157) 00010000:00010000:3.0:1787784886.344657:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 10 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.344677:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 10 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784886.344686:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9895063370c0 00000100:00100000:3.0:1787784886.344719:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 16 00000100:00100000:3.0:1787784886.344729:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658275456:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:3.0:1787784886.345669:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063370c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658275456:12345-192.168.203.43@tcp:4:cp.0 Request processed in 936us (1167us total) trans 4294967374 rc 0/0 00000100:00100000:3.0:1787784886.345680:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 16 00002000:00100000:3.0:1787784886.345690:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1c:0x0] (0->157) req@ffff9895063370c0 x1874627658275456/t4294967374(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:12/0 lens 648/448 e 0 to 0 dl 1787784897 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784886.345699:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895063370c0 with x1874627658275456 ext(0->157) 00010000:00010000:3.0:1787784886.345705:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 12 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.345719:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 12 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784886.345726:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9895063370c0 00000100:00100000:0.0:1787784886.353717:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784886.354661:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658275584 00000100:00100000:2.0:1787784886.354677:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658275584 00010000:00010000:2.0:1787784886.354729:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 12 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784886.354781:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 431 00000100:00100000:2.0:1787784886.354792:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658275584:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784886.354811:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784886.354839:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 12 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.354856:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 12 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.354898:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc774d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1792 expref: 12 pid: 6671 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.354919:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784886.354923:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784886.354930:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.354942:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.354974:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784886.355039:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658275584:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 249us (1335us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784886.355048:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 431 00010000:00010000:2.0:1787784886.355078:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355088:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355114:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355124:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355133:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355142:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784886.355170:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355181:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355195:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.355202:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008481b064/0xb0ce63137acc77a1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784886.355255:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658275200:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 34175us (34571us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784886.355265:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 409 00000100:00100000:0.0:1787784886.363553:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.363819:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658275840 00000100:00100000:3.0:1787784886.363834:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658275840 00000100:00100000:3.0:1787784886.363904:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1440 00000100:00100000:3.0:1787784886.363916:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32787:x1874627658275840:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784886.364039:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32787:x1874627658275840:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 126us (498us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.364048:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1440 00000100:00100000:0.0:1787784886.370926:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.372595:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658275968 00000100:00100000:3.0:1787784886.372610:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658275968 00000100:00100000:3.0:1787784886.372738:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 410 00000100:00100000:3.0:1787784886.372749:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658275968:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784886.372768:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784886.372787:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17bc expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784886.372814:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1c:0x0] initial LVB size: 158, mtime: 0x6a8f6eb4, atime: 0x0, ctime: 0x6a8f6eb4, blocks: 0x1 00010000:00010000:3.0:1787784886.372833:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=677s v=(0 0) 00010000:00010000:3.0:1787784886.372838:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784886.372848:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.372864:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 11 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.372894:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784886.372957:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658275968:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 211us (2047us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.372964:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 410 00000020:00080000:3.0:1787784886.380242:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 34 batchid updated 00000001:00080000:3.0:1787784886.380270:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968252 is committed 00000001:00080000:3.0:1787784886.380293:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968254 is committed 00000020:00080000:3.0:1787784886.380389:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 34 00000020:00080000:3.0:1787784886.380402:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 34 00000100:00100000:3.0:1787784886.387333:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784886.389422:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658276224 00000100:00100000:0.0:1787784886.389443:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658276224 00000100:00100000:0.0:1787784886.389551:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1441 00000100:00100000:0.0:1787784886.389564:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32787:x1874627658276224:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784886.389713:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32787:x1874627658276224:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 152us (2390us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784886.389722:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1441 00000100:00100000:1.0:1787784886.404660:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784886.405042:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658276352 00000100:00100000:2.0:1787784886.405056:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658276352 00002000:00100000:2.0:1787784886.405123:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1c:0x0] (0->4095) req@ffff989540666d00 x1874627658276352/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:12/0 lens 488/0 e 0 to 0 dl 1787784897 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784886.405182:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540666d00 with x1874627658276352 ext(0->4095) 00002000:00010000:2.0:1787784886.405190:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989540666d00 00000100:00100000:2.0:1787784886.405222:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 122 00000100:00100000:2.0:1787784886.405713:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658276352:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784886.406192:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658276352:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 912us (1498us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784886.406206:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 122 00002000:00100000:2.0:1787784886.406218:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1c:0x0] (0->4095) req@ffff989540666d00 x1874627658276352/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:12/0 lens 488/4536 e 0 to 0 dl 1787784897 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784886.406228:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540666d00 with x1874627658276352 ext(0->4095) 00002000:00010000:2.0:1787784886.406234:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989540666d00 00000100:00100000:0.0:1787784886.418810:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784886.418949:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658276480 00000100:00100000:1.0:1787784886.418962:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658276480 00000100:00100000:1.0:1787784886.419051:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1442 00000100:00100000:1.0:1787784886.419064:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32787:x1874627658276480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784886.419091:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784886.419118:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc77af lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17c3 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.419209:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.419227:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.419251:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.419453:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc77af lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 54 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.419476:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.419648:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:1.0:1787784886.419651:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784886.419655:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3e00, rc 0) 00000100:00100000:1.0:1787784886.419725:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658276480:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 663us (929us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784886.419732:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1442 00000100:00100000:2.0:1787784886.437241:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784886.437394:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658276608 00000100:00100000:1.0:1787784886.437405:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658276608 00000100:00100000:1.0:1787784886.437462:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1443 00000100:00100000:1.0:1787784886.437474:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658276608:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784886.437510:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784886.437612:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658276608:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 139us (379us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784886.437620:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1443 00000100:00100000:1.0:1787784886.446560:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.452801:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658276736 00000100:00100000:3.0:1787784886.452815:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658276736 00000100:00100000:1.0:1787784886.460815:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 411 00000100:00100000:1.0:1787784886.460832:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658276736:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784886.461064:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461082:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784886.461087:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784886.461093:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461107:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461139:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461173:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461183:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461389:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784886.461395:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461405:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461421:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.461428:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e175fd38/0xb0ce63137acc77bd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784886.461517:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658276736:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 689us (14963us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784886.461526:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 411 00000100:00100000:1.0:1787784886.464083:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.464197:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658276992 00000100:00100000:3.0:1787784886.464213:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658276992 00000100:00100000:1.0:1787784886.464371:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1444 00000100:00100000:1.0:1787784886.464388:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658276992:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784886.464461:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.464478:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.464488:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.464526:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784886.464547:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 53 pid: 20438 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784886.464562:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539131e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668447488:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784886.465317:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784886.475245:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.475302:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658277120 00000100:00100000:3.0:1787784886.475314:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658277120 00010000:00010000:3.0:1787784886.475367:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 54 pid: 20438 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784886.475419:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 432 00000100:00100000:3.0:1787784886.475429:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658277120:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784886.475523:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784886.475532:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 54 pid: 20438 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.475549:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 54 pid: 20438 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.475576:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784886.475582:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.475593:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.475604:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.475624:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784886.475686:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658277120:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 259us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.475695:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 432 00010000:00010000:0.0:1787784886.487457:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc77b6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17c3 expref: 53 pid: 20438 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784886.487494:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539131e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668447488:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784886.487529:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.487542:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.487549:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.487823:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.487831:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.487838:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.487844:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784886.487848:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.487874:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.488215:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:0.0:1787784886.488218:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784886.488223:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.488230:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77c4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784886.488528:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658276992:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 24142us (24451us total) trans 4294968256 rc 0/0 00000100:00100000:0.0:1787784886.488539:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1444 00000100:00100000:0.0:1787784886.497440:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784886.498204:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658277376 00000100:00100000:2.0:1787784886.498219:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658277376 00000100:00100000:2.0:1787784886.498743:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 290 00000100:00100000:2.0:1787784886.498757:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32787:x1874627658277376:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000040:00080000:1.0:1787784886.502806:0:22488:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784886.502820:0:22488:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:2.0:1787784886.503375:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32787:x1874627658277376:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 4612us (5940us total) trans 4294968257 rc 0/0 00000100:00100000:2.0:1787784886.503391:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 290 00000040:00080000:1.0:1787784886.503818:0:22488:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 44 count 3 00000040:00080000:1.0:1787784886.503829:0:22488:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784886.595583:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.595677:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658277504 00000100:00100000:3.0:1787784886.595689:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658277504 00000100:00100000:3.0:1787784886.595797:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 412 00000100:00100000:3.0:1787784886.595809:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658277504:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784886.595833:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784886.595866:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17ca expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.596050:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784886.596055:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784886.596068:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.596091:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.596108:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784886.596223:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658277504:12345-192.168.203.43@tcp:101:rm.0 Request processed in 416us (654us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.596236:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 412 00000100:00100000:0.0:1787784886.612212:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784886.612379:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658277760 00000100:00100000:3.0:1787784886.612393:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658277760 00000100:00100000:3.0:1787784886.612463:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1445 00000100:00100000:3.0:1787784886.612475:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32789:x1874627658277760:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:3.0:1787784886.612548:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1787784886.612582:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7769 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1799 expref: 107 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.612606:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784886.612619:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc777e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17a7 expref: 106 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.612628:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784886.612632:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784886.612661:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.612677:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.612688:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.612703:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.612710:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.612718:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.613351:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.613367:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 53 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.613376:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.613420:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 53 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.613436:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 53 pid: 5813 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784886.613454:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668447616:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784886.613572:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ceaa80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784886.618642:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784886.618683:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658277888 00000100:00100000:2.0:1787784886.618694:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658277888 00010000:00010000:2.0:1787784886.618747:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 54 pid: 5813 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784886.618793:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 433 00000100:00100000:2.0:1787784886.618804:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658277888:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784886.618822:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784886.618831:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 54 pid: 5813 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.618846:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 54 pid: 5813 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.619271:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784886.619278:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.619291:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.619300:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.619324:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784886.619429:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658277888:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 625us (800us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784886.619438:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 433 00010000:00010000:3.0:1787784886.619660:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7738 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1784 expref: 53 pid: 5813 timeout: 777 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784886.619679:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668447616:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784886.619708:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.619718:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.619725:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784886.620913:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].14, 64 off9112 00000004:00080000:3.0:1787784886.620928:0:5812:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 677707779765ns, next at 678707776686ns, rc = 0 00010000:00010000:3.0:1787784886.621004:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621024:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621037:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621046:0:5812:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621053:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621060:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621087:0:5812:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787784886.621116:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621122:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621129:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621136:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784886.621141:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621178:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621191:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:3.0:1787784886.621194:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784886.621199:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621205:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc77e0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621228:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621233:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621239:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621244:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784886.621247:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621253:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621267:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784886.621269:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784886.621273:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.621278:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc77d2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784886.621290:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f2b2a941 reply state 0000000014bf47d2 transno 4294968258 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784886.621300:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784886.621387:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32789:x1874627658277760:12345-192.168.203.43@tcp:36:rm.0 Request processed in 8911us (9186us total) trans 4294968258 rc 0/0 00000100:00100000:3.0:1787784886.621396:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1445 00010000:00010000:2.0:1787784886.621788:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784886.621796:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.621804:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784886.621808:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.621815:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.621831:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff9895198b5000) 00010000:00010000:2.0:1787784886.621833:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787784886.621847:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.621854:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc77e7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784886.621871:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784886.621883:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784886.621885:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784886.639416:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.639439:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.639447:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.639454:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784886.639459:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.639468:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.639495:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784886.639499:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784886.639505:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784886.639512:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc77d9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784886.771140:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668448128:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.772100:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784886.772211:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1787784886.772220:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668447744:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.772815:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784886.772850:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668448256:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.773379:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784886.773439:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403124f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668447872:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.774650:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784886.774700:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668448384:0@lo:400:kworker.0 00000100:00100000:1.0:1787784886.775865:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668448256 00000100:00100000:1.0:1787784886.776033:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668448256 00000100:00100000:1.0:1787784886.776177:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668447872 00000100:00100000:1.0:1787784886.776182:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668447872 00000100:00100000:1.0:1787784886.776210:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:1.0:1787784886.776221:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668448256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784886.776327:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668448256:12345-0@lo:400:kworker.0 Request processed in 108us (2948us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784886.776335:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:1.0:1787784886.776356:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:1.0:1787784886.776362:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668447872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784886.776395:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668447872:12345-0@lo:400:kworker.0 Request processed in 33us (1752us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784886.776400:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:1.0:1787784886.776457:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668448256:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.778807:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784886.781011:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403124f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668447872:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781054:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1787784886.781076:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668448000:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781179:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784886.781228:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668448512:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784886.781257:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784886.781316:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668448128 00000100:00100000:3.0:1787784886.781325:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668448128 00000100:00080000:0.0:1787784886.781370:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784886.781388:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668448384 00000100:00100000:3.0:1787784886.781391:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668448384 00000100:00100000:3.0:1787784886.781417:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:3.0:1787784886.781426:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668448128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781497:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668448128:12345-0@lo:400:kworker.0 Request processed in 71us (9400us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.781504:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:3.0:1787784886.781523:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1787784886.781530:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668448384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781566:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668448384:12345-0@lo:400:kworker.0 Request processed in 37us (2765us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.781572:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1787784886.781593:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668447744 00000100:00100000:3.0:1787784886.781597:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668447744 00000100:00100000:0.0:1787784886.781614:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668448128:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781624:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 612 00000100:00100000:3.0:1787784886.781630:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668447744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781674:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668447744:12345-0@lo:400:kworker.0 Request processed in 44us (8860us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.781681:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 612 00000100:00100000:0.0:1787784886.781693:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668447744:0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781710:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668448000 00000100:00100000:3.0:1787784886.781714:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668448000 00000100:00100000:3.0:1787784886.781747:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 488 00000100:00100000:3.0:1787784886.781755:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668448000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784886.781786:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668448000:12345-0@lo:400:kworker.0 Request processed in 33us (608us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784886.781795:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 488 00000100:00100000:2.0:1787784886.783560:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668448512 00000100:00100000:2.0:1787784886.783573:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668448512 00000100:00100000:2.0:1787784886.783621:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1787784886.783630:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668448512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784886.783674:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784886.783749:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668448512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 120us (2490us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784886.783756:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1787784886.783821:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668448000:0@lo:400:kworker.0 00000100:00100000:2.0:1787784886.783841:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668448512:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784886.783851:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668448384:0@lo:400:kworker.0 00000100:00080000:1.0:1787784886.785413:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784886.785425:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784886.873486:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784886.879252:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658278016 00000100:00100000:0.0:1787784886.879275:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658278016 00000100:00100000:0.0:1787784886.879365:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1446 00000100:00100000:0.0:1787784886.879377:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+106:32805:x1874627658278016:12345-192.168.203.43@tcp:101:mkdir.0 00010000:00010000:0.0:1787784886.879408:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784886.879436:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc77ee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17d1 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.879539:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc77f5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.879557:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc77f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.879570:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc77f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784886.879645:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 7680 00000004:00200000:0.0:1787784886.879649:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784886.879675:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc77f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.879707:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc77ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17d1 expref: 108 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.879731:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc77f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17d1 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784886.879741:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784886.879744:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784886.879748:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00100000:0.0:1787784886.879839:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32805:x1874627658278016:12345-192.168.203.43@tcp:101:mkdir.0 Request processed in 463us (6368us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784886.879857:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1446 00000100:00100000:0.0:1787784887.000804:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.000981:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658278144 00000100:00100000:1.0:1787784887.000994:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658278144 00000100:00100000:1.0:1787784887.001092:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1447 00000100:00100000:1.0:1787784887.001106:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32811:x1874627658278144:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784887.001139:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.001205:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77fc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17d8 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.001410:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77fc lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17d8 expref: 54 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.001429:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff98951b73b300) 00010000:00010000:1.0:1787784887.001432:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.001439:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc77fc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17d8 expref: 54 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.001464:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403640400, rc 0) 00000100:00100000:1.0:1787784887.001556:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32811:x1874627658278144:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 455us (767us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784887.001566:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1447 00000100:00080000:2.0:1787784887.001741:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784887.020632:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.020754:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658278400 00000100:00100000:2.0:1787784887.020766:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658278400 00000100:00080000:1.0:1787784887.021098:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784887.021337:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1448 00000100:00100000:2.0:1787784887.021355:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:32811:x1874627658278400:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.021383:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.021387:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787784887.021426:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc77f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17d1 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.021453:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.021459:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784887.021476:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7803 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17df expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.021654:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.021671:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.021682:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.021697:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.021704:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.021710:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.026895:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.026922:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.026934:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.026951:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 0 00010000:00010000:2.0:1787784887.026998:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027006:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027013:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027018:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.027022:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027029:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027041:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.027043:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.027048:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027054:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc780a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784887.027070:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000074a2d8b2 reply state 00000000ea61bb75 transno 4294968259 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.027078:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784887.027122:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7803 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 109 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027136:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 108 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.027143:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784887.027169:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784887.027173:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac200, rc 0) 00000100:00100000:2.0:1787784887.027243:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:32811:x1874627658278400:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 5893us (6624us total) trans 4294968259 rc 0/0 00000100:00100000:2.0:1787784887.027884:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1448 00000100:00100000:3.0:1787784887.033536:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784887.033536:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.033558:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.033569:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.033576:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.033584:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.033594:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.033615:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520131800) 00010000:00010000:1.0:1787784887.033619:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.033625:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.033631:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7811 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.034124:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658278656 00000100:00100000:2.0:1787784887.034135:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658278656 00000100:00100000:2.0:1787784887.034207:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1449 00000100:00100000:2.0:1787784887.034217:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32811:x1874627658278656:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.034240:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.034261:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7826 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17e6 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.034358:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.034373:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 109 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.034387:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 4/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.034420:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 109 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.034433:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 109 pid: 5812 timeout: 778 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.034447:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894011ed2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668448640:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784887.034534:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784887.043138:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.043793:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894011ed2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668448640:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784887.043819:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784887.045420:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658278784 00000100:00100000:0.0:1787784887.045439:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658278784 00010000:00010000:0.0:1787784887.045511:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 110 pid: 5812 timeout: 778 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784887.045562:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 434 00000100:00100000:0.0:1787784887.045573:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:2357:x1874627658278784:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784887.045591:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784887.045599:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 110 pid: 5812 timeout: 778 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.045615:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 110 pid: 5812 timeout: 778 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.045641:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7818 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17df expref: 110 pid: 5812 timeout: 778 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.045661:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784887.045667:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.045677:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.045686:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.045717:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784887.045797:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:2357:x1874627658278784:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 224us (2667us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784887.045810:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 434 00010000:00010000:2.0:1787784887.045816:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.045832:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.045854:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 0 00010000:00010000:2.0:1787784887.046137:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046179:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046191:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046197:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.046201:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046208:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046222:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784887.046225:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784887.046230:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 2/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046236:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc782d lrc: 0/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046307:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7826 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17e6 expref: 109 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046319:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.046321:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.046325:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7826 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17e6 expref: 109 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.046334:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16a00, rc 0) 00000100:00100000:2.0:1787784887.046395:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32811:x1874627658278656:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 12178us (12869us total) trans 4294968260 rc 301/301 00000100:00100000:2.0:1787784887.046407:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1449 00000100:00100000:3.0:1787784887.055460:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.055560:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658279040 00000100:00100000:0.0:1787784887.055578:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658279040 00000100:00100000:0.0:1787784887.055657:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:0.0:1787784887.055670:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32811:x1874627658279040:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784887.056229:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:32811:x1874627658279040:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 559us (776us total) trans 4294968261 rc 0/0 00000100:00100000:0.0:1787784887.056239:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 291 00000100:00100000:1.0:1787784887.059129:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.059353:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658279168 00000100:00100000:0.0:1787784887.059366:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658279168 00000100:00100000:0.0:1787784887.059443:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1450 00000100:00100000:0.0:1787784887.059454:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:32811:x1874627658279168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784887.059486:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.059516:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc783b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17ed expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.059594:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc783b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ed expref: 110 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.059611:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc783b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ed expref: 110 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.059636:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520108a00) 00010000:00010000:0.0:1787784887.059639:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787784887.059644:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b800, rc 0) 00000100:00100000:0.0:1787784887.059718:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:32811:x1874627658279168:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 264us (605us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784887.059727:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1450 00000100:00100000:1.0:1787784887.064575:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.065380:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658279424 00000100:00100000:2.0:1787784887.065390:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658279424 00000100:00100000:2.0:1787784887.065455:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1451 00000100:00100000:2.0:1787784887.065465:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508341c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:32811:x1874627658279424:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.065531:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.065547:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.065557:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.066940:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.066957:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.066964:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.066970:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.066974:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.066981:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.067002:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520108a00) 00010000:00010000:2.0:1787784887.067005:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787784887.067010:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.067017:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc7842 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.067108:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508341c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:32811:x1874627658279424:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 1640us (2539us total) trans 4294968262 rc 0/0 00000100:00100000:2.0:1787784887.067120:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1451 00000100:00100000:1.0:1787784887.076711:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.076901:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658279680 00000100:00100000:2.0:1787784887.076911:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658279680 00000100:00100000:0.0:1787784887.077001:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1452 00000100:00100000:0.0:1787784887.077026:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:32811:x1874627658279680:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784887.077476:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.077500:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.077515:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787784887.077605:0:5811:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968263 00000004:00010000:0.0:1787784887.077654:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e9580676 reply state 00000000eb8f7995 transno 4294968264 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784887.077664:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784887.077803:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:32811:x1874627658279680:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 786us (1098us total) trans 4294968264 rc 0/0 00000100:00100000:0.0:1787784887.077811:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1452 00000020:00080000:0.0:1787784887.077819:0:5811:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1787784887.088403:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.088429:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.088438:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.088445:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.088452:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.088461:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.088486:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520108a00) 00010000:00010000:3.0:1787784887.088490:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787784887.088496:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.088503:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7849 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784887.091437:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.092114:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658279936 00000100:00100000:3.0:1787784887.092125:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658279936 00000100:00100000:3.0:1787784887.092213:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 292 00000100:00100000:3.0:1787784887.092226:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:32811:x1874627658279936:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784887.092480:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:32811:x1874627658279936:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 255us (1051us total) trans 4294968265 rc 0/0 00000100:00100000:3.0:1787784887.092488:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 292 00000100:00100000:1.0:1787784887.172002:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.172131:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658280192 00000100:00100000:3.0:1787784887.172141:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658280192 00000100:00100000:3.0:1787784887.172235:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1453 00000100:00100000:3.0:1787784887.172245:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:32814:x1874627658280192:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784887.172268:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.172287:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7850 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17f4 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172356:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172370:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172380:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172393:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172400:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172406:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172534:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7865 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172555:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7865 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172564:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7865 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.172574:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.172664:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172672:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172678:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172684:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.172688:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172695:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172706:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.172710:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.172714:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172724:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7857 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172736:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172742:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172748:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172753:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.172756:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172761:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172768:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.172770:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.172773:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172779:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc785e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172815:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7850 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17f4 expref: 117 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172833:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7865 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17f4 expref: 116 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.172840:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520108a00) 00010000:00010000:3.0:1787784887.172843:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784887.172846:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784887.172919:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:32814:x1874627658280192:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 675us (926us total) trans 4294968266 rc 0/0 00000100:00100000:3.0:1787784887.172928:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1453 00000100:00100000:1.0:1787784887.187856:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.188183:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658280320 00000100:00100000:3.0:1787784887.188193:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658280320 00000100:00100000:3.0:1787784887.188245:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1454 00000100:00100000:3.0:1787784887.188255:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:32814:x1874627658280320:12345-192.168.203.43@tcp:54:lfs.0 00010000:00010000:3.0:1787784887.188332:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188350:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188361:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188388:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188396:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188406:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188412:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.188416:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188422:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188433:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520108a00) 00010000:00010000:3.0:1787784887.188436:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.188440:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.188445:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7873 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784887.188518:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:32814:x1874627658280320:12345-192.168.203.43@tcp:54:lfs.0 Request processed in 264us (671us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784887.188526:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1454 00000100:00100000:1.0:1787784887.194810:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.194903:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658280576 00000100:00100000:2.0:1787784887.194913:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658280576 00000100:00100000:2.0:1787784887.194978:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 293 00000100:00100000:2.0:1787784887.194989:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:32814:x1874627658280576:12345-192.168.203.43@tcp:35:lfs.0 00000100:00100000:2.0:1787784887.195350:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32814:x1874627658280576:12345-192.168.203.43@tcp:35:lfs.0 Request processed in 362us (548us total) trans 4294968267 rc 0/0 00000100:00100000:2.0:1787784887.195358:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 293 00010000:00010000:1.0:1787784887.218737:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784887.276809:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.277132:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658280832 00000100:00100000:0.0:1787784887.277142:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658280832 00000100:00100000:0.0:1787784887.277226:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1455 00000100:00100000:0.0:1787784887.277237:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32821:x1874627658280832:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784887.277262:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.277353:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc787a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b17fb expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.277473:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7881 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.277488:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7881 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.277505:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7881 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784887.277516:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1787784887.277730:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc787a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17fb expref: 120 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.277750:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7881 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17fb expref: 119 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.277758:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520108a00) 00010000:00010000:0.0:1787784887.277761:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1787784887.277764:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00100000:0.0:1787784887.277928:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:32821:x1874627658280832:12345-192.168.203.43@tcp:101:cat.0 Request processed in 692us (1127us total) trans 4294968268 rc 0/0 00000100:00100000:0.0:1787784887.277936:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1455 00000100:00100000:1.0:1787784887.294607:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.294693:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658281344 00000100:00100000:2.0:1787784887.294709:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658281344 00000100:00100000:2.0:1787784887.294782:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 67 00000100:00100000:2.0:1787784887.294792:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:32821:x1874627658281344:12345-192.168.203.43@tcp:103:cat.0 00010000:00010000:2.0:1787784887.294810:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787784887.294835:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7865 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17f4 expref: 119 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.294862:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7881 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17fb expref: 118 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.294894:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc783b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ed expref: 117 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.294904:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520108a00) 00010000:00010000:2.0:1787784887.294918:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784887.294980:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:32821:x1874627658281344:12345-192.168.203.43@tcp:103:cat.0 Request processed in 190us (384us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784887.294987:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 67 00000100:00100000:2.0:1787784887.306019:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.306058:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.306088:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658281472 00000100:00100000:2.0:1787784887.306097:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658281472 00000100:00100000:2.0:1787784887.306167:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658281088 00000100:00100000:2.0:1787784887.306171:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658281088 00000100:00100000:2.0:1787784887.306196:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1456 00000100:00100000:2.0:1787784887.306205:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32821:x1874627658281472:12345-192.168.203.43@tcp:58:cat.0 00010000:00010000:2.0:1787784887.306307:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.306324:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.306335:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784887.306725:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1457 00000040:00080000:2.0:1787784887.306857:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].44, 136 off14176 00000100:00100000:3.0:1787784887.306942:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:32823:x1874627658281088:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.306954:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.306970:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7896 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1802 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.307038:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc789d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.307050:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc789d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.307059:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc789d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.307069:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.307181:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7896 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1802 expref: 120 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.307198:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7400/0xb0ce63137acc789d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1802 expref: 119 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.307206:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.307209:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787784887.307212:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7400, rc 0) 00000100:00100000:3.0:1787784887.307268:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:32823:x1874627658281088:12345-192.168.203.43@tcp:101:cat.0 Request processed in 333us (1209us total) trans 4294968269 rc 0/0 00000100:00100000:3.0:1787784887.307276:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1457 00000100:00100000:2.0:1787784887.307914:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:32821:x1874627658281472:12345-192.168.203.43@tcp:58:cat.0 Request processed in 1707us (1901us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784887.307922:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1456 00000100:00100000:2.0:1787784887.310222:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.310323:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658281728 00000100:00100000:0.0:1787784887.310331:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658281728 00000100:00100000:0.0:1787784887.310404:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1458 00000100:00100000:0.0:1787784887.310421:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:32822:x1874627658281728:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784887.310438:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.310455:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78ab lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1809 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310529:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310542:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310553:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310566:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310572:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310579:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310701:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310710:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310716:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784887.310724:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1787784887.310814:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310825:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310831:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310837:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.310841:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310848:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310859:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784887.310862:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784887.310866:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310886:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310901:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310907:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310913:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310918:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.310921:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310926:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310933:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.310936:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.310938:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310944:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310953:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310959:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310965:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310970:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.310973:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310978:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310984:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520108400) 00010000:00010000:0.0:1787784887.310987:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.310990:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.310995:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.311033:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78ab lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1809 expref: 120 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.311043:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.311046:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784887.311049:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78ab lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1809 expref: 120 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.311060:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a200, rc 0) 00000100:00100000:0.0:1787784887.311113:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:32822:x1874627658281728:12345-192.168.203.43@tcp:101:cat.0 Request processed in 701us (897us total) trans 4294968270 rc 301/301 00000100:00100000:0.0:1787784887.311120:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1458 00000100:00100000:3.0:1787784887.316071:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.316202:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658281856 00000100:00100000:1.0:1787784887.316212:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658281856 00000100:00100000:1.0:1787784887.316263:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1459 00000100:00100000:1.0:1787784887.316272:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:32821:x1874627658281856:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.316291:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.316310:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1810 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.316362:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.316372:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 4/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1810 expref: 120 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.316388:0:5813:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.316400:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 3/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1810 expref: 120 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.316423:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.316427:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787784887.316430:0:5813:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 3/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1810 expref: 120 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.316437:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403641600, rc 0) 00000100:00100000:1.0:1787784887.316492:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+120:32821:x1874627658281856:12345-192.168.203.43@tcp:101:cat.0 Request processed in 221us (431us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784887.316499:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1459 00000100:00100000:1.0:1787784887.323016:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.323248:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.323373:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658282112 00000100:00100000:2.0:1787784887.323382:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658282112 00000100:00100000:2.0:1787784887.323429:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 294 00000100:00100000:2.0:1787784887.323438:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+120:32822:x1874627658282112:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.323975:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658282368 00000100:00100000:0.0:1787784887.323980:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658282368 00000100:00100000:0.0:1787784887.324065:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1460 00000100:00100000:0.0:1787784887.324073:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+122:32825:x1874627658282368:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784887.324084:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.324100:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78d5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1817 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324178:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324195:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324206:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324219:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324225:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324231:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324338:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324348:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324355:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784887.324365:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1787784887.324422:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324430:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324437:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324444:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.324448:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324474:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324486:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784887.324489:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784887.324493:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324499:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc78ea lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324513:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324518:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324523:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324528:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403351800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.324530:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324536:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324546:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.324548:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.324551:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324557:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351800/0xb0ce63137acc78dc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324565:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324571:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324577:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324582:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.324585:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324590:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324615:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520108400) 00010000:00010000:0.0:1787784887.324618:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.324622:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324640:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc78e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324677:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78d5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1817 expref: 124 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324687:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.324690:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784887.324693:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403351a00/0xb0ce63137acc78d5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1817 expref: 124 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.324702:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:0.0:1787784887.324739:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+123:32825:x1874627658282368:12345-192.168.203.43@tcp:101:cat.0 Request processed in 656us (1481us total) trans 4294968272 rc 301/301 00000100:00100000:0.0:1787784887.324747:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1460 00000100:00100000:2.0:1787784887.324799:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+121:32822:x1874627658282112:12345-192.168.203.43@tcp:35:cat.0 Request processed in 428us (857us total) trans 4294968271 rc 0/0 00000100:00100000:2.0:1787784887.324809:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 294 00000100:00100000:1.0:1787784887.331439:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.331516:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658282624 00000100:00100000:0.0:1787784887.331527:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658282624 00000100:00100000:0.0:1787784887.331582:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1461 00000100:00100000:0.0:1787784887.331592:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+122:32819:x1874627658282624:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784887.331643:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.331664:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78f8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b181e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331751:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331765:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331776:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331790:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331796:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331802:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331941:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331951:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.331957:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784887.331966:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1787784887.332051:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332058:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332065:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332071:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.332075:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332087:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332099:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784887.332102:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784887.332106:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332112:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc790d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332128:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332134:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332140:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332389:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.332394:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332401:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332409:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.332412:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.332415:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332424:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc78ff lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332433:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332439:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332446:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332451:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.332453:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332459:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332466:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520108000) 00010000:00010000:0.0:1787784887.332469:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.332472:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.332477:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7906 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784887.333029:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.333539:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658282880 00000100:00100000:3.0:1787784887.333546:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658282880 00000100:00100000:3.0:1787784887.333591:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 295 00000100:00100000:3.0:1787784887.333602:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+125:32825:x1874627658282880:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.333985:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.334072:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658283008 00000100:00100000:1.0:1787784887.334077:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658283008 00000100:00100000:1.0:1787784887.334115:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1462 00000100:00100000:1.0:1787784887.334122:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+126:32822:x1874627658283008:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.334132:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.334174:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc791b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1825 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.334245:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.334260:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.334270:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.334293:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00000004:00200000:1.0:1787784887.334296:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784887.334311:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.334335:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc791b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 128 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.334357:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 127 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.334365:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.334368:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.334371:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3a00, rc 0) 00000100:00100000:1.0:1787784887.334421:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+127:32822:x1874627658283008:12345-192.168.203.43@tcp:101:cat.0 Request processed in 300us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784887.334428:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1462 00000100:00100000:3.0:1787784887.335236:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+127:32825:x1874627658282880:12345-192.168.203.43@tcp:35:cat.0 Request processed in 1634us (2206us total) trans 4294968274 rc 0/0 00000100:00100000:3.0:1787784887.335249:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 295 00010000:00010000:2.0:1787784887.340568:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78f8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b181e expref: 126 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.340590:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.340593:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.340598:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc78f8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b181e expref: 126 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.340618:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b800, rc 0) 00000100:00100000:2.0:1787784887.340701:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+125:32819:x1874627658282624:12345-192.168.203.43@tcp:101:cat.0 Request processed in 9109us (9268us total) trans 4294968273 rc 301/301 00000100:00100000:2.0:1787784887.340710:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1461 00000100:00100000:3.0:1787784887.347910:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.347954:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.348000:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658283520 00000100:00100000:3.0:1787784887.348010:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658283520 00000100:00100000:3.0:1787784887.348058:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00100000:3.0:1787784887.348069:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+125:32823:x1874627658283520:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.348316:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+126:32823:x1874627658283520:12345-192.168.203.43@tcp:35:cat.0 Request processed in 249us (360us total) trans 4294968275 rc 0/0 00000100:00100000:3.0:1787784887.348324:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 296 00000100:00100000:2.0:1787784887.348708:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658283264 00000100:00100000:2.0:1787784887.348713:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658283264 00000100:00100000:2.0:1787784887.348751:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1463 00000100:00100000:2.0:1787784887.348759:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+126:32827:x1874627658283264:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784887.348770:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.348788:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7929 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b182c expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.348846:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.348860:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.348881:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.348914:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.348922:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.348928:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349051:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349061:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349068:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.349078:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1787784887.349135:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349142:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349168:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349174:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.349178:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349186:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349198:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784887.349201:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784887.349206:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349212:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc793e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349226:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349231:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349240:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349244:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.349246:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349251:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349257:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.349259:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.349262:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349267:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7930 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349274:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349280:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349285:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349289:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.349291:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349299:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349304:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520131c00) 00010000:00010000:2.0:1787784887.349306:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.349308:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349313:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7937 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349344:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7929 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b182c expref: 128 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349354:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.349356:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.349360:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7929 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b182c expref: 128 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.349368:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16400, rc 0) 00000100:00100000:2.0:1787784887.349398:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+127:32827:x1874627658283264:12345-192.168.203.43@tcp:101:cat.0 Request processed in 641us (1498us total) trans 4294968276 rc 301/301 00000100:00100000:2.0:1787784887.349405:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1463 00000100:00100000:0.0:1787784887.367969:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.368292:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658284032 00000100:00100000:3.0:1787784887.368302:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658284032 00000100:00100000:3.0:1787784887.368361:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 297 00000100:00100000:3.0:1787784887.368371:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+127:32819:x1874627658284032:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.368612:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+128:32819:x1874627658284032:12345-192.168.203.43@tcp:35:cat.0 Request processed in 241us (649us total) trans 4294968277 rc 0/0 00000100:00100000:3.0:1787784887.368620:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 297 00000100:00100000:3.0:1787784887.376070:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.376183:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658283776 00000100:00100000:1.0:1787784887.376193:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658283776 00000100:00100000:1.0:1787784887.376256:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1464 00000100:00100000:1.0:1787784887.376267:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+128:32824:x1874627658283776:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.376287:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.376307:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc794c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1833 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376385:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376400:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376412:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376471:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376481:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376489:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376627:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376637:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376645:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.376654:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.376750:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376758:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376764:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376771:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.376775:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376783:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376798:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.376801:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.376806:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376811:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7961 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376828:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376834:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376839:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376844:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.376847:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376853:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376861:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.376863:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.376866:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376884:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7953 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376896:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376903:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376909:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376914:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.376917:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376923:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376930:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b300) 00010000:00010000:1.0:1787784887.376932:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.376934:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376940:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc795a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376977:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc794c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1833 expref: 130 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.376988:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.376990:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.376995:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc794c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1833 expref: 130 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.377005:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953f5b3400, rc 0) 00000100:00100000:1.0:1787784887.377095:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+129:32824:x1874627658283776:12345-192.168.203.43@tcp:101:cat.0 Request processed in 830us (1034us total) trans 4294968278 rc 301/301 00000100:00100000:1.0:1787784887.377104:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1464 00000100:00100000:0.0:1787784887.383469:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.383503:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.383769:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658284288 00000100:00100000:2.0:1787784887.383778:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658284288 00000100:00100000:2.0:1787784887.383820:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1465 00000100:00100000:2.0:1787784887.383828:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+129:32829:x1874627658284288:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784887.383854:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.383889:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc796f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b183a expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.383952:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.383965:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.383975:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.383988:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.383995:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384001:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384111:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384120:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384126:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.384135:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1787784887.384240:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384248:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384255:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384260:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.384264:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384272:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384284:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784887.384287:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784887.384291:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384298:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7984 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384312:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384317:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384324:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384333:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.384336:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384342:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384348:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.384350:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.384352:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384357:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7976 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384364:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384370:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384375:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384380:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.384382:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384388:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384398:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520131c00) 00010000:00010000:2.0:1787784887.384400:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.384403:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384408:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc797d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384439:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc796f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b183a expref: 131 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384448:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.384450:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.384463:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc796f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b183a expref: 131 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.384472:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad000, rc 0) 00000100:00100000:2.0:1787784887.384523:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+130:32829:x1874627658284288:12345-192.168.203.43@tcp:101:cat.0 Request processed in 695us (1019us total) trans 4294968279 rc 301/301 00000100:00100000:2.0:1787784887.384530:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1465 00000100:00100000:3.0:1787784887.385638:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658284544 00000100:00100000:3.0:1787784887.385646:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658284544 00000100:00100000:3.0:1787784887.385693:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 298 00000100:00100000:3.0:1787784887.385702:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+130:32824:x1874627658284544:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.385856:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+131:32824:x1874627658284544:12345-192.168.203.43@tcp:35:cat.0 Request processed in 154us (2396us total) trans 4294968280 rc 0/0 00000100:00100000:3.0:1787784887.385884:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 298 00000100:00100000:2.0:1787784887.413736:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.413785:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.413910:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658284800 00000100:00100000:0.0:1787784887.413921:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658284800 00000100:00100000:0.0:1787784887.414001:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1466 00000100:00100000:0.0:1787784887.414011:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+131:32836:x1874627658284800:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784887.414037:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.414057:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7992 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1841 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414134:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414181:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414195:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414229:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414236:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414244:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414373:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414383:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414391:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784887.414401:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1787784887.414494:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414503:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414509:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414515:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.414520:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414527:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414539:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784887.414542:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784887.414818:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414826:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc79a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414845:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414854:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414859:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414863:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.414867:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414884:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414893:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.414895:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.414898:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414903:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7999 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414911:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414917:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414934:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414940:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.414943:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414949:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414956:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520108000) 00010000:00010000:0.0:1787784887.414959:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.414961:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.414967:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc79a0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.415008:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7992 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1841 expref: 133 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.415018:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.415020:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784887.415023:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7992 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1841 expref: 133 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.415032:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbc400, rc 0) 00000100:00100000:0.0:1787784887.415094:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+132:32836:x1874627658284800:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1083us (1307us total) trans 4294968281 rc 301/301 00000100:00100000:0.0:1787784887.415101:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1466 00000100:00100000:3.0:1787784887.415483:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658285056 00000100:00100000:3.0:1787784887.415491:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658285056 00000100:00100000:3.0:1787784887.415533:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 299 00000100:00100000:3.0:1787784887.415541:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+132:32827:x1874627658285056:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.415713:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+133:32827:x1874627658285056:12345-192.168.203.43@tcp:35:cat.0 Request processed in 172us (1986us total) trans 4294968282 rc 0/0 00000100:00100000:3.0:1787784887.415720:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 299 00000100:00100000:0.0:1787784887.441759:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.441878:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658285312 00000100:00100000:1.0:1787784887.441888:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658285312 00000100:00100000:1.0:1787784887.441949:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1467 00000100:00100000:1.0:1787784887.441959:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+133:32835:x1874627658285312:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.441986:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.442006:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc79b5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1848 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442103:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442117:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442129:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442143:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442196:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442207:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442372:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442382:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442390:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.442399:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.442486:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442494:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442501:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442507:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.442511:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442520:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442533:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.442536:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.442541:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442547:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8e00/0xb0ce63137acc79ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442563:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442570:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442576:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442581:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.442584:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442590:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442598:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.442600:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.442604:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442610:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc79bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442618:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442624:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442631:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442637:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.442640:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442656:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442664:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b300) 00010000:00010000:1.0:1787784887.442666:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.442668:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442674:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9a00/0xb0ce63137acc79c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442711:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc79b5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1848 expref: 135 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442720:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.442722:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.442725:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9c00/0xb0ce63137acc79b5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1848 expref: 135 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.442733:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9c00, rc 0) 00000100:00100000:1.0:1787784887.442812:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+134:32835:x1874627658285312:12345-192.168.203.43@tcp:101:cat.0 Request processed in 855us (1063us total) trans 4294968283 rc 301/301 00000100:00100000:1.0:1787784887.442823:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1467 00000100:00100000:2.0:1787784887.445554:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.446244:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658285568 00000100:00100000:3.0:1787784887.446274:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658285568 00000100:00100000:3.0:1787784887.446334:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:3.0:1787784887.446343:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+134:32829:x1874627658285568:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.446583:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+135:32829:x1874627658285568:12345-192.168.203.43@tcp:35:cat.0 Request processed in 240us (1035us total) trans 4294968284 rc 0/0 00000100:00100000:3.0:1787784887.446592:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 300 00000100:00100000:3.0:1787784887.453935:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.453978:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.454054:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658286080 00000100:00100000:3.0:1787784887.454063:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658286080 00000100:00100000:3.0:1787784887.454108:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 301 00000100:00100000:3.0:1787784887.454117:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+135:32836:x1874627658286080:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.454368:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+136:32836:x1874627658286080:12345-192.168.203.43@tcp:35:cat.0 Request processed in 251us (389us total) trans 4294968285 rc 0/0 00000100:00100000:3.0:1787784887.454377:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 301 00000100:00100000:1.0:1787784887.454471:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658285824 00000100:00100000:1.0:1787784887.454475:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658285824 00000100:00100000:1.0:1787784887.454525:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1468 00000100:00100000:1.0:1787784887.454533:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+136:32828:x1874627658285824:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.454554:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.454573:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8a00/0xb0ce63137acc79d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b184f expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454632:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454646:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454657:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454672:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454678:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454685:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454815:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454829:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454837:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.454845:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.454908:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454915:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454922:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454927:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.454930:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454937:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454948:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.454951:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.454956:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454962:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc79ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454981:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454988:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454994:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.454998:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.455001:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455007:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455013:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.455016:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.455019:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455025:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc79df lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455049:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455055:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455060:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455065:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.455067:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455072:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455079:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.455081:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.455083:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455088:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc79e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455120:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8a00/0xb0ce63137acc79d8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b184f expref: 138 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455130:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.455132:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.455136:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8a00/0xb0ce63137acc79d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b184f expref: 138 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.455144:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8a00, rc 0) 00000100:00100000:1.0:1787784887.455196:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+137:32828:x1874627658285824:12345-192.168.203.43@tcp:101:cat.0 Request processed in 665us (1280us total) trans 4294968286 rc 301/301 00000100:00100000:1.0:1787784887.455204:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1468 00000100:00100000:2.0:1787784887.470588:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.470658:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.470858:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658286592 00000100:00100000:1.0:1787784887.470886:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658286592 00000100:00100000:1.0:1787784887.470964:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1469 00000100:00100000:1.0:1787784887.470977:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+137:32826:x1874627658286592:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.471010:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.471035:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8a00/0xb0ce63137acc79fb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1856 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471133:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471190:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471204:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471231:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471239:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471246:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471421:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471432:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471440:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.471451:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.471585:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471594:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471600:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471607:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.471611:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471620:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471634:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.471638:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.471643:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471650:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7a10 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471670:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471676:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471686:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471691:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.471694:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471700:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471707:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.471709:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.471712:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471718:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7a02 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471727:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471733:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471739:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471745:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.471748:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471759:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471766:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.471768:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.471770:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471776:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7a09 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471824:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8a00/0xb0ce63137acc79fb lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1856 expref: 139 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471836:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.471838:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.471842:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8a00/0xb0ce63137acc79fb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1856 expref: 139 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.471853:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8a00, rc 0) 00000100:00100000:1.0:1787784887.471941:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+138:32826:x1874627658286592:12345-192.168.203.43@tcp:101:cat.0 Request processed in 966us (1281us total) trans 4294968287 rc 301/301 00000100:00100000:1.0:1787784887.471950:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1469 00000100:00100000:0.0:1787784887.472627:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658286336 00000100:00100000:0.0:1787784887.472636:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658286336 00000100:00100000:0.0:1787784887.472685:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 302 00000100:00100000:0.0:1787784887.472696:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+138:32835:x1874627658286336:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.472887:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+139:32835:x1874627658286336:12345-192.168.203.43@tcp:35:cat.0 Request processed in 192us (2314us total) trans 4294968288 rc 0/0 00000100:00100000:0.0:1787784887.472894:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 302 00000100:00100000:2.0:1787784887.483472:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.483648:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658286848 00000100:00100000:0.0:1787784887.483661:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658286848 00000100:00100000:0.0:1787784887.483922:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 303 00000100:00100000:0.0:1787784887.483935:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+139:32826:x1874627658286848:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.484443:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+140:32826:x1874627658286848:12345-192.168.203.43@tcp:35:cat.0 Request processed in 509us (993us total) trans 4294968289 rc 0/0 00000100:00100000:0.0:1787784887.484452:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 303 00000100:00100000:2.0:1787784887.489442:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.491191:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658287104 00000100:00100000:3.0:1787784887.491211:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658287104 00000100:00100000:3.0:1787784887.491315:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1470 00000100:00100000:3.0:1787784887.491327:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+140:32832:x1874627658287104:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.491354:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.491381:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a1e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b185d expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.491499:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.491518:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.491825:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.491845:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.491854:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.491861:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.492047:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:3.0:1787784887.492990:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493006:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493020:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.493032:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.493167:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493175:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493184:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493191:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.493196:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493205:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493219:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.493222:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.493227:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493234:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7a33 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493253:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493263:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493268:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493273:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.493276:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493281:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493288:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.493290:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.493293:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493297:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a25 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493306:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493312:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493318:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493323:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.493327:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493333:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493340:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.493342:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.493344:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493350:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a2c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493402:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a1e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b185d expref: 142 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493412:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.493414:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.493417:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a1e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b185d expref: 142 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.493426:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784887.493513:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+141:32832:x1874627658287104:12345-192.168.203.43@tcp:101:cat.0 Request processed in 2187us (4086us total) trans 4294968290 rc 301/301 00000100:00100000:3.0:1787784887.493523:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1470 00000100:00100000:1.0:1787784887.497346:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658287360 00000100:00100000:1.0:1787784887.497366:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658287360 00000100:00100000:1.0:1787784887.497664:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:1.0:1787784887.497741:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+141:32828:x1874627658287360:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:1.0:1787784887.498356:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+142:32828:x1874627658287360:12345-192.168.203.43@tcp:35:cat.0 Request processed in 679us (6307us total) trans 4294968291 rc 0/0 00000100:00100000:1.0:1787784887.498366:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 304 00000100:00100000:0.0:1787784887.509410:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.509620:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658287616 00000100:00100000:3.0:1787784887.509632:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658287616 00000100:00100000:3.0:1787784887.509771:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1471 00000100:00100000:3.0:1787784887.509784:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+142:32844:x1874627658287616:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.509814:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.509842:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7a41 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1864 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.510011:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.510030:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.510042:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.510057:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.510063:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.510070:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.512074:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.512101:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.512114:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.512128:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.513092:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513106:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513114:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513121:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.513126:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513134:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513192:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.513196:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.513201:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513208:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7a56 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513239:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513245:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513251:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513260:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.513264:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513270:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513279:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.513281:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.513284:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513290:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7a48 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513300:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513307:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513323:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513329:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.513332:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513339:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513351:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.513354:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.513357:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513363:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7a4f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513435:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7a41 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1864 expref: 144 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513447:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.513450:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.513454:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7a41 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1864 expref: 144 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.513464:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:3.0:1787784887.515069:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+143:32844:x1874627658287616:12345-192.168.203.43@tcp:101:cat.0 Request processed in 5275us (5660us total) trans 4294968292 rc 301/301 00000100:00100000:3.0:1787784887.515086:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1471 00000100:00100000:0.0:1787784887.515649:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.516446:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658287872 00000100:00100000:1.0:1787784887.516460:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658287872 00000100:00100000:0.0:1787784887.517756:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:0.0:1787784887.517780:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+143:32832:x1874627658287872:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.518106:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+144:32832:x1874627658287872:12345-192.168.203.43@tcp:35:cat.0 Request processed in 336us (2458us total) trans 4294968293 rc 0/0 00000100:00100000:0.0:1787784887.518115:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 305 00000100:00100000:0.0:1787784887.523286:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.523381:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658288128 00000100:00100000:2.0:1787784887.523394:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658288128 00000100:00100000:2.0:1787784887.523479:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1472 00000100:00100000:2.0:1787784887.523492:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+144:32848:x1874627658288128:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784887.523536:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.523563:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7a64 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b186b expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523662:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523682:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523695:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523711:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523718:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523726:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523909:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523920:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.523928:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.523938:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1787784887.524049:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524057:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524064:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524071:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.524076:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524084:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524097:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784887.524100:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784887.524105:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524111:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a79 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524130:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524140:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524175:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524181:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.524184:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524190:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524198:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.524200:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.524203:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524208:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a6b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524217:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524224:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524230:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524235:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.524239:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524246:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524254:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520131c00) 00010000:00010000:2.0:1787784887.524256:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.524258:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524263:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a72 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524309:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7a64 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b186b expref: 146 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524335:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.524337:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.524341:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7a64 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b186b expref: 146 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.524351:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac600, rc 0) 00000100:00100000:2.0:1787784887.524432:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+145:32848:x1874627658288128:12345-192.168.203.43@tcp:101:cat.0 Request processed in 942us (1160us total) trans 4294968294 rc 301/301 00000100:00100000:2.0:1787784887.524440:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1472 00000100:00100000:2.0:1787784887.526301:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.526622:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658288384 00000100:00100000:3.0:1787784887.526634:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658288384 00000100:00100000:3.0:1787784887.526699:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00100000:3.0:1787784887.526710:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+145:32844:x1874627658288384:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.528390:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+146:32844:x1874627658288384:12345-192.168.203.43@tcp:35:cat.0 Request processed in 1669us (2088us total) trans 4294968295 rc 0/0 00000100:00100000:2.0:1787784887.528410:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 306 00000100:00100000:1.0:1787784887.533076:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.533190:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658288640 00000100:00100000:2.0:1787784887.533204:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658288640 00000100:00100000:2.0:1787784887.533281:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1473 00000100:00100000:2.0:1787784887.533294:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+146:32842:x1874627658288640:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784887.533339:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.533368:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7a87 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1872 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533469:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533490:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533515:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533532:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533539:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533546:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533708:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533725:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533733:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.533744:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1787784887.533857:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533866:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533881:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533889:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.533894:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533904:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533917:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784887.533920:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784887.533926:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533932:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7a9c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533959:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533966:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533971:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533976:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.533979:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533986:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.533993:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.533996:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.533999:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534006:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7a8e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534016:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534023:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534029:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534036:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.534039:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534045:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534053:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520131c00) 00010000:00010000:2.0:1787784887.534055:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.534058:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534064:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7a95 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534112:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7a87 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1872 expref: 148 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534124:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.534126:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.534130:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7a87 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1872 expref: 148 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.534140:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac600, rc 0) 00000100:00100000:2.0:1787784887.534296:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+147:32842:x1874627658288640:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1004us (1233us total) trans 4294968296 rc 301/301 00000100:00100000:2.0:1787784887.534304:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1473 00000100:00100000:0.0:1787784887.536919:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.537005:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658288896 00000100:00100000:1.0:1787784887.537016:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658288896 00000100:00100000:1.0:1787784887.537071:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 307 00000100:00100000:1.0:1787784887.537080:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+147:32848:x1874627658288896:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:1.0:1787784887.537402:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+148:32848:x1874627658288896:12345-192.168.203.43@tcp:35:cat.0 Request processed in 321us (493us total) trans 4294968297 rc 0/0 00000100:00100000:1.0:1787784887.537411:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 307 00000040:00080000:0.0:1787784887.538003:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784887.538009:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784887.538049:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 45 count 4 00000040:00080000:0.0:1787784887.538055:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784887.538191:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668448768:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1787784887.538272:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4480 going to sleep for 11 seconds 00000100:00100000:2.0:1787784887.545723:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668448768:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784887.547886:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.548324:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658289152 00000100:00100000:1.0:1787784887.548338:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658289152 00000100:00100000:1.0:1787784887.548400:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 308 00000100:00100000:1.0:1787784887.548412:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+148:32842:x1874627658289152:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:1.0:1787784887.548726:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+149:32842:x1874627658289152:12345-192.168.203.43@tcp:35:cat.0 Request processed in 315us (862us total) trans 4294968298 rc 0/0 00000100:00100000:1.0:1787784887.548735:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 308 00000100:00100000:2.0:1787784887.557026:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.557999:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658289408 00000100:00100000:3.0:1787784887.558015:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658289408 00000100:00100000:3.0:1787784887.558085:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1474 00000100:00100000:3.0:1787784887.558096:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+149:32839:x1874627658289408:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.558126:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.558174:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7aaa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1879 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.558277:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.558297:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.558311:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.558341:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.558349:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.558355:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.558947:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.558982:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658289536 00000100:00100000:2.0:1787784887.558987:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658289536 00000100:00100000:2.0:1787784887.559025:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1475 00000100:00100000:2.0:1787784887.559034:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658289536:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784887.559057:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x52:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784887.559201:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658289536:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 167us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784887.559209:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1475 00010000:00010000:3.0:1787784887.560051:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.560071:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.560083:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.562032:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.562217:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562226:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562233:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562239:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.562244:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562252:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562272:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.562275:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.562281:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562288:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7abf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562311:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562331:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562341:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562346:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.562349:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562356:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562364:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.562367:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.562369:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562374:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7ab1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562384:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562389:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562395:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562400:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.562402:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562412:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562419:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5c00) 00010000:00010000:3.0:1787784887.562421:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.562423:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.562429:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7ab8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.568218:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7aaa lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1879 expref: 151 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.568245:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.568248:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.568254:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc7aaa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1879 expref: 151 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.568275:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:2.0:1787784887.568378:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+150:32839:x1874627658289408:12345-192.168.203.43@tcp:101:cat.0 Request processed in 10282us (11367us total) trans 4294968299 rc 301/301 00000100:00100000:2.0:1787784887.568389:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1474 00000100:00100000:2.0:1787784887.570517:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.570987:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658289664 00000100:00100000:1.0:1787784887.571001:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658289664 00000100:00100000:1.0:1787784887.571074:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1476 00000100:00100000:1.0:1787784887.571086:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658289664:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1787784887.571205:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571227:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571241:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571259:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571267:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571274:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571302:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571442:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571453:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.571847:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00000004:00200000:1.0:1787784887.571854:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784887.571943:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571959:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571969:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571977:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571983:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.571987:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.571995:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572010:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:1.0:1787784887.572012:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.572017:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572023:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8c00/0xb0ce63137acc7acd lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572039:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572044:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572050:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572054:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.572058:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572064:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572070:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x8138294e (ffff98951b73b200) 00010000:00010000:1.0:1787784887.572072:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.572075:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572080:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8600/0xb0ce63137acc7ad4 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572096:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572101:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572107:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572110:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.572113:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572119:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572125:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.572127:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.572130:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.572136:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7adb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784887.572263:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658289664:12345-192.168.203.43@tcp:34:lhsmtool_posix.0 Request processed in 1177us (1767us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784887.572274:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1476 00000100:00100000:2.0:1787784887.576329:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.576372:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.576607:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658290176 00000100:00100000:3.0:1787784887.576619:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658290176 00000100:00100000:3.0:1787784887.576689:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1477 00000100:00100000:3.0:1787784887.576700:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+150:32834:x1874627658290176:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.576732:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.576760:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7ae2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1880 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.576860:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.576892:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.576906:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.576922:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.576930:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.576936:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.577096:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.577107:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.577114:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.577125:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.577286:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.577295:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.577303:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.577310:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00000100:00100000:1.0:1787784887.578988:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658289920 00000100:00100000:1.0:1787784887.579006:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658289920 00000100:00100000:1.0:1787784887.579084:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 309 00000100:00100000:1.0:1787784887.579095:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+153:32839:x1874627658289920:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:1.0:1787784887.579798:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+154:32839:x1874627658289920:12345-192.168.203.43@tcp:35:cat.0 Request processed in 701us (3490us total) trans 4294968301 rc 0/0 00000100:00100000:1.0:1787784887.579809:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 309 00010000:00010000:3.0:1787784887.582474:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582505:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582536:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.582540:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.582547:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582556:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7af7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582588:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582596:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582603:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582610:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.582613:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582619:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582627:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.582630:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.582632:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582637:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ae9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582651:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582656:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582661:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582665:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.582667:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582682:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582690:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.582692:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.582695:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.582701:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7af0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.585529:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7ae2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1880 expref: 153 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.585554:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.585557:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.585564:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7ae2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1880 expref: 153 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.585587:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:3.0:1787784887.587693:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+152:32834:x1874627658290176:12345-192.168.203.43@tcp:101:cat.0 Request processed in 10969us (11294us total) trans 4294968300 rc 301/301 00000100:00100000:3.0:1787784887.587710:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1477 00000100:00100000:2.0:1787784887.592266:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.593017:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658290432 00000100:00100000:3.0:1787784887.593030:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658290432 00000100:00100000:3.0:1787784887.593113:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1478 00000100:00100000:3.0:1787784887.593125:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658290432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784887.593183:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.593211:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7b05 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1887 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593345:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593364:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593377:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.593391:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.593526:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593535:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593546:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593553:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.593558:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593566:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593581:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.593585:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.593590:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593597:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc7b0c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593649:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7b05 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1887 expref: 55 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593662:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff98950a38c800) 00010000:00010000:3.0:1787784887.593664:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.593669:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc7b05 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1887 expref: 55 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.593679:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6800, rc 0) 00000100:00100000:3.0:1787784887.593754:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32851:x1874627658290432:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 630us (1501us total) trans 4294968302 rc 301/301 00000100:00100000:3.0:1787784887.593766:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1478 00000100:00100000:1.0:1787784887.601958:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.602023:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.602194:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658291072 00000100:00100000:3.0:1787784887.602208:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658291072 00000100:00100000:3.0:1787784887.602283:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1479 00000100:00100000:3.0:1787784887.602296:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32851:x1874627658291072:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784887.602358:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x52:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784887.602515:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32851:x1874627658291072:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 221us (491us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784887.602525:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1479 00000100:00100000:2.0:1787784887.602985:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658290688 00000100:00100000:2.0:1787784887.602991:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658290688 00000100:00100000:2.0:1787784887.603029:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00100000:2.0:1787784887.603036:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+152:32834:x1874627658290688:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.603282:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+153:32834:x1874627658290688:12345-192.168.203.43@tcp:35:cat.0 Request processed in 245us (1340us total) trans 4294968303 rc 0/0 00000100:00100000:2.0:1787784887.603291:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 310 00000100:00100000:1.0:1787784887.611072:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.612709:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658290944 00000100:00100000:3.0:1787784887.612726:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658290944 00000100:00100000:3.0:1787784887.612809:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1480 00000100:00100000:3.0:1787784887.612820:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+153:32831:x1874627658290944:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.612848:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.612891:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7b1a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b188e expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.612995:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613013:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613027:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613045:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613053:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613058:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613245:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613256:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613264:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.613275:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.613410:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613418:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613425:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613431:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.613436:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613443:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613471:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.613474:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.613480:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.613486:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615212:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615230:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615237:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615243:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.615248:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615257:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615274:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.615277:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.615281:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615288:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b21 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615310:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615317:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615322:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615328:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.615330:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615336:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615344:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.615356:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.615359:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615365:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b28 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615436:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7b1a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b188e expref: 155 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615448:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.615454:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.615460:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7b1a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b188e expref: 155 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.615472:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784887.615937:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+154:32831:x1874627658290944:12345-192.168.203.43@tcp:101:cat.0 Request processed in 3115us (4885us total) trans 4294968304 rc 301/301 00000100:00100000:3.0:1787784887.615948:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1480 00000100:00100000:3.0:1787784887.618921:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.619142:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658291328 00000100:00100000:1.0:1787784887.619185:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658291328 00000100:00100000:1.0:1787784887.619254:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1481 00000100:00100000:1.0:1787784887.619266:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32851:x1874627658291328:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784887.619294:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.619333:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7b3d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1895 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784887.623815:0:20438:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1f:0x0], index 1: 0 00010000:00010000:2.0:1787784887.625649:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7b44 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.625681:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7b44 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.625696:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7b44 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.625794:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9200/0xb0ce63137acc7b3d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1895 expref: 57 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.625817:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7b44 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1895 expref: 56 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.625826:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131c00) 00010000:00010000:2.0:1787784887.625830:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784887.625834:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989503c16000, rc 0) 00000100:00100000:2.0:1787784887.625945:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32851:x1874627658291328:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 6680us (7034us total) trans 4294968305 rc 0/0 00000100:00100000:2.0:1787784887.625960:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1481 00000100:00100000:0.0:1787784887.632486:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.632660:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.633319:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658291584 00000100:00100000:2.0:1787784887.633333:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658291584 00000100:00100000:2.0:1787784887.633405:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 311 00000100:00100000:2.0:1787784887.633424:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+154:32831:x1874627658291584:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.633827:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658291840 00000100:00100000:3.0:1787784887.633832:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658291840 00000100:00100000:3.0:1787784887.633890:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1482 00000100:00100000:2.0:1787784887.633999:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+156:32831:x1874627658291584:12345-192.168.203.43@tcp:35:cat.0 Request processed in 576us (1337us total) trans 4294968306 rc 0/0 00000100:00100000:2.0:1787784887.634008:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 311 00000100:00100000:3.0:1787784887.634010:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+156:32840:x1874627658291840:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.634023:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.634050:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7b52 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b189c expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634126:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634143:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634177:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634193:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634201:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634211:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634349:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634360:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634367:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.634378:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.634438:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634448:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634455:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634462:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.634466:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634475:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634490:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.634493:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.634510:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634517:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7b67 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634535:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634541:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634546:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634551:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.634554:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634560:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634568:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.634571:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.634574:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634580:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7b59 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634588:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634594:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634601:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634606:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.634608:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634614:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634620:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.634623:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.634626:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634632:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7b60 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634671:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7b52 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b189c expref: 157 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634683:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.634685:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.634689:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7b52 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b189c expref: 157 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.634699:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:2.0:1787784887.635766:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+156:32840:x1874627658291840:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1862us (3290us total) trans 4294968307 rc 301/301 00000100:00100000:2.0:1787784887.635777:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1482 00000100:00100000:3.0:1787784887.643776:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.643827:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.643878:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.643882:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658292096 00000100:00100000:2.0:1787784887.643895:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658292096 00000100:00100000:2.0:1787784887.643962:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658292352 00000100:00100000:2.0:1787784887.643965:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658292352 00000100:00100000:2.0:1787784887.643998:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1483 00000100:00100000:2.0:1787784887.644009:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32851:x1874627658292096:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.644087:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784887.644124:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7b44 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1895 expref: 56 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644181:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131c00) 00010000:00010000:2.0:1787784887.644196:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784887.644237:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644259:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644270:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644440:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644463:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644476:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644560:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644567:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644574:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644580:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.644584:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644591:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644603:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.644607:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.644611:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644619:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7b75 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644637:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644643:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644650:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644657:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ac600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.644661:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784887.644668:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1484 00010000:00010000:2.0:1787784887.644675:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.644687:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0xef48fa17 (ffff989520131800) 00010000:00010000:2.0:1787784887.644693:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.644696:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784887.644699:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+156:32833:x1874627658292352:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784887.644718:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7b7c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0xef48fa17 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.644725:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.644743:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7b83 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18a3 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.644809:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.644820:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.644829:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.644841:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32851:x1874627658292096:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 827us (1072us total) trans 0 rc -2/-2 00010000:00010000:0.0:1787784887.644847:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.644857:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1483 00010000:00010000:0.0:1787784887.644861:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.644882:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.644930:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658292608 00000100:00100000:2.0:1787784887.644940:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658292608 00000100:00100000:2.0:1787784887.644992:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 312 00000100:00100000:2.0:1787784887.645002:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+158:32840:x1874627658292608:12345-192.168.203.43@tcp:35:cat.0 00010000:00010000:0.0:1787784887.645034:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645048:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645058:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784887.645069:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1787784887.645212:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645230:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645237:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645243:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.645247:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645254:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645266:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784887.645269:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784887.645272:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645279:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc7b98 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645294:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645304:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645309:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645314:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.645317:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645323:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645331:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.645333:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.645336:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645342:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd000/0xb0ce63137acc7b8a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645351:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645357:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645362:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645368:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.645371:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645377:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645384:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520108000) 00010000:00010000:0.0:1787784887.645386:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.645389:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645394:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7b91 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.645585:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+160:32840:x1874627658292608:12345-192.168.203.43@tcp:35:cat.0 Request processed in 582us (1713us total) trans 4294968308 rc 0/0 00000100:00100000:2.0:1787784887.645594:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 312 00010000:00010000:0.0:1787784887.645628:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7b83 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18a3 expref: 159 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645641:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.645644:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784887.645648:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7b83 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18a3 expref: 159 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.645658:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:0.0:1787784887.645708:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+158:32833:x1874627658292352:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1017us (1880us total) trans 4294968309 rc 301/301 00000100:00100000:0.0:1787784887.645716:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1484 00000100:00100000:3.0:1787784887.648479:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.649209:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658292736 00000100:00100000:2.0:1787784887.649225:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658292736 00000100:00100000:2.0:1787784887.649311:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1485 00000100:00100000:2.0:1787784887.649321:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+55:32851:x1874627658292736:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.649339:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.649372:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7ba6 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18aa expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.649406:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.649419:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.649439:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.649547:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7ba6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 57 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.649572:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 56 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.649581:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:2.0:1787784887.649583:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784887.649586:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ad000, rc 0) 00000100:00100000:2.0:1787784887.649653:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32851:x1874627658292736:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 333us (1199us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784887.649661:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1485 00000100:00100000:1.0:1787784887.662606:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.662840:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658292992 00000100:00100000:3.0:1787784887.662853:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658292992 00000100:00100000:3.0:1787784887.662947:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1486 00000100:00100000:3.0:1787784887.662959:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+158:32845:x1874627658292992:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.662992:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.663024:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7bb4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18b1 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663128:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663175:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663188:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663206:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663213:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663220:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663391:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663407:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663414:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.663425:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.663558:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663567:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663575:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663582:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.663587:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663596:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663610:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.663614:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.663620:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663626:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7bc9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663650:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663655:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663661:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663665:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.663668:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663673:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663681:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.663683:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.663686:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663692:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7bbb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663700:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663705:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663710:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663716:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.663719:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663725:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663733:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.663735:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.663738:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663744:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7bc2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663794:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7bb4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18b1 expref: 160 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663805:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.663807:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.663810:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7bb4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18b1 expref: 160 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.663820:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784887.663918:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+159:32845:x1874627658292992:12345-192.168.203.43@tcp:101:cat.0 Request processed in 960us (1326us total) trans 4294968310 rc 301/301 00000100:00100000:3.0:1787784887.663927:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1486 00000100:00100000:1.0:1787784887.667367:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.667422:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.667841:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658293248 00000100:00100000:3.0:1787784887.667854:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658293248 00000100:00100000:3.0:1787784887.667938:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 313 00000100:00100000:3.0:1787784887.667950:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+159:32833:x1874627658293248:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.668269:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+160:32833:x1874627658293248:12345-192.168.203.43@tcp:35:cat.0 Request processed in 319us (916us total) trans 4294968311 rc 0/0 00000100:00100000:3.0:1787784887.668279:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 313 00000100:00100000:3.0:1787784887.675389:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.675519:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658293504 00000100:00100000:2.0:1787784887.675533:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658293504 00000100:00100000:2.0:1787784887.675603:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658293760 00000100:00100000:2.0:1787784887.675606:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658293760 00000100:00100000:2.0:1787784887.675637:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1487 00000100:00100000:2.0:1787784887.675648:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32851:x1874627658293504:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.675768:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.675789:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.675803:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784887.676350:0:5811:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].19, 88 off9528 00000004:00080000:2.0:1787784887.676363:0:5811:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 678763215286ns, next at 679763212259ns, rc = 0 00000004:00010000:2.0:1787784887.676471:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e5f4651f reply state 000000005d39e7e2 transno 4294968312 ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.676483:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784887.676546:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1488 00000100:00100000:0.0:1787784887.676559:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+160:32837:x1874627658293760:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784887.676576:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.676591:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc7bde lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18b8 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.676599:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+58:32851:x1874627658293504:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 953us (9176us total) trans 4294968312 rc 0/0 00000100:00100000:2.0:1787784887.676607:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1487 00010000:00010000:0.0:1787784887.676656:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676668:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676677:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676692:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676699:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676706:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676821:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676831:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676839:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784887.676848:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1787784887.676923:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676935:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676942:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676949:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.676954:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676963:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676978:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:0.0:1787784887.676981:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784887.676987:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.676994:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7bf3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677013:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677019:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677024:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677029:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbce00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.677032:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677037:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677044:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.677046:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.677049:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677054:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbce00/0xb0ce63137acc7be5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677062:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677067:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677072:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677077:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784887.677079:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677085:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677092:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520108000) 00010000:00010000:0.0:1787784887.677094:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784887.677098:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677103:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7bec lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677174:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc7bde lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18b8 expref: 162 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677185:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:0.0:1787784887.677187:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787784887.677191:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc7bde lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18b8 expref: 162 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.677200:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989401dbd000, rc 0) 00000100:00100000:0.0:1787784887.677227:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+161:32837:x1874627658293760:12345-192.168.203.43@tcp:101:cat.0 Request processed in 672us (1852us total) trans 4294968313 rc 301/301 00000100:00100000:0.0:1787784887.677233:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1488 00010000:00010000:3.0:1787784887.682600:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.682618:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.682627:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.682635:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.682641:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.682651:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.682682:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:3.0:1787784887.682685:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787784887.682692:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.682699:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7bd7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784887.683108:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.683202:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.683286:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658294144 00000100:00100000:3.0:1787784887.683299:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658294144 00000100:00100000:3.0:1787784887.683379:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 314 00000100:00100000:3.0:1787784887.683390:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+161:32845:x1874627658294144:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:3.0:1787784887.683717:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+162:32845:x1874627658294144:12345-192.168.203.43@tcp:35:cat.0 Request processed in 328us (515us total) trans 4294968314 rc 0/0 00000100:00100000:3.0:1787784887.683726:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 314 00000100:00100000:2.0:1787784887.684178:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658293888 00000100:00100000:2.0:1787784887.684187:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658293888 00000100:00100000:2.0:1787784887.684225:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1489 00000100:00100000:2.0:1787784887.684235:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+57:32851:x1874627658293888:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.684253:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.684277:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18bf expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.684350:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 58 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.684367:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 58 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.684389:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:2.0:1787784887.684393:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784887.684397:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac600, rc 0) 00000100:00100000:2.0:1787784887.684438:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+58:32851:x1874627658293888:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 206us (1955us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784887.684445:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1489 00000100:00100000:1.0:1787784887.691486:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.691546:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.691703:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658294656 00000100:00100000:2.0:1787784887.691716:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658294656 00000100:00100000:2.0:1787784887.691790:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:2.0:1787784887.691802:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+162:32837:x1874627658294656:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.692115:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+163:32837:x1874627658294656:12345-192.168.203.43@tcp:35:cat.0 Request processed in 315us (567us total) trans 4294968315 rc 0/0 00000100:00100000:2.0:1787784887.692123:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 315 00000100:00100000:3.0:1787784887.692238:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658294400 00000100:00100000:3.0:1787784887.692243:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658294400 00000100:00100000:3.0:1787784887.692278:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1490 00000100:00100000:3.0:1787784887.692286:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+163:32850:x1874627658294400:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.692298:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.692323:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7c08 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18c6 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692404:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692422:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692437:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692453:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692460:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692467:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692620:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692632:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692641:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.692651:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.692722:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692731:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692738:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692744:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.692749:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692757:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692771:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.692774:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.692781:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692788:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7c1d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692807:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692812:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692818:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692822:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.692825:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692830:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692837:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.692839:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.692842:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692848:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7c0f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692857:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692863:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692879:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692884:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.692887:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692893:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692899:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.692901:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.692904:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692910:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7c16 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692947:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7c08 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18c6 expref: 165 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692958:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.692960:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.692963:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7c08 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18c6 expref: 165 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.692972:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784887.693021:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+164:32850:x1874627658294400:12345-192.168.203.43@tcp:101:cat.0 Request processed in 736us (1550us total) trans 4294968316 rc 301/301 00000100:00100000:3.0:1787784887.693032:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1490 00000100:00100000:3.0:1787784887.696032:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.696121:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658294784 00000100:00100000:2.0:1787784887.696133:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658294784 00000100:00100000:2.0:1787784887.696246:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 413 00000100:00100000:2.0:1787784887.696257:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658294784:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1787784887.696685:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658294784:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 429us (667us total) trans 4294967386 rc 0/0 00000100:00100000:2.0:1787784887.696694:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 413 00000100:00080000:1.0:1787784887.696856:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784887.699476:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.699543:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.699699:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658295040 00000100:00100000:1.0:1787784887.699713:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658295040 00000100:00100000:1.0:1787784887.699789:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1491 00000100:00100000:1.0:1787784887.699800:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+164:32820:x1874627658295040:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.699831:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.699859:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7c2b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18cd expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.699982:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700000:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700013:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700031:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700037:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700043:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700242:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700253:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700260:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.700271:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.700379:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700386:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700392:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700397:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.700401:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700408:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700421:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.700424:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.700429:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700440:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c40 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700458:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700463:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700468:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700472:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.700474:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700479:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700485:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.700487:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.700490:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700495:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c32 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700639:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700646:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700653:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700658:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.700661:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700667:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700674:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.700677:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.700679:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700685:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c39 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700730:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7c2b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18cd expref: 166 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700741:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.700743:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.700747:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7c2b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18cd expref: 166 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.700756:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8800, rc 0) 00000100:00100000:1.0:1787784887.700835:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+165:32820:x1874627658295040:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1037us (1383us total) trans 4294968317 rc 301/301 00000100:00100000:1.0:1787784887.700848:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1491 00000100:00100000:2.0:1787784887.701780:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658295296 00000100:00100000:2.0:1787784887.701796:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658295296 00000100:00100000:2.0:1787784887.701850:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 316 00000100:00100000:2.0:1787784887.701859:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d1c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+165:32850:x1874627658295296:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.702730:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d1c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+166:32850:x1874627658295296:12345-192.168.203.43@tcp:35:cat.0 Request processed in 866us (3180us total) trans 4294968318 rc 0/0 00000100:00100000:2.0:1787784887.702744:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 316 00000100:00100000:1.0:1787784887.708447:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.708516:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.708671:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658295552 00000100:00100000:2.0:1787784887.708683:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658295552 00000100:00100000:2.0:1787784887.708756:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1492 00000100:00100000:2.0:1787784887.708769:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+166:32847:x1874627658295552:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784887.708804:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.708834:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7c4e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18d4 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.708948:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.708967:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.708980:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.708997:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709005:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709011:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709215:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709227:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709233:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.709244:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1787784887.709355:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709368:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709375:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709382:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.709387:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709396:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709411:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784887.709415:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784887.709420:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709427:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16000/0xb0ce63137acc7c63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709446:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709452:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709459:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709464:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894010ad800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.709468:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709474:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709482:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.709485:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.709487:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709493:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad800/0xb0ce63137acc7c55 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709625:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709633:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709638:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709644:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784887.709647:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709653:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709662:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff989520131800) 00010000:00010000:2.0:1787784887.709665:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784887.709668:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709681:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16400/0xb0ce63137acc7c5c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709727:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7c4e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18d4 expref: 168 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709740:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:2.0:1787784887.709743:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787784887.709746:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac200/0xb0ce63137acc7c4e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18d4 expref: 168 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.709756:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894010ac200, rc 0) 00000100:00100000:2.0:1787784887.709833:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+167:32847:x1874627658295552:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1066us (1401us total) trans 4294968319 rc 301/301 00000100:00100000:2.0:1787784887.709842:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1492 00000100:00100000:3.0:1787784887.710322:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658295808 00000100:00100000:3.0:1787784887.710330:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658295808 00000100:00100000:3.0:1787784887.710593:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 317 00000100:00100000:3.0:1787784887.710604:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+58:32851:x1874627658295808:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784887.710822:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+59:32851:x1874627658295808:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 220us (2305us total) trans 4294968320 rc 0/0 00000100:00100000:3.0:1787784887.710831:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 317 00000100:00100000:1.0:1787784887.716879:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.716934:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.716960:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.716995:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658296320 00000100:00100000:1.0:1787784887.717006:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658296320 00000100:00100000:1.0:1787784887.717057:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658296576 00000100:00100000:1.0:1787784887.717059:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658296576 00000100:00100000:1.0:1787784887.717081:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1493 00000100:00100000:1.0:1787784887.717090:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+167:32830:x1874627658296320:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.717121:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1787784887.717525:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658296064 00000100:00100000:2.0:1787784887.717531:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658296064 00000100:00100000:2.0:1787784887.717561:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 318 00000100:00100000:2.0:1787784887.717567:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+168:32820:x1874627658296064:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.717843:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+169:32820:x1874627658296064:12345-192.168.203.43@tcp:35:cat.0 Request processed in 275us (991us total) trans 4294968321 rc 0/0 00000100:00100000:2.0:1787784887.717851:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 318 00010000:00010000:1.0:1787784887.718234:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7c71 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18db expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.719070:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.719092:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.719106:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.719121:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.719129:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.719136:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784887.720469:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1494 00000100:00100000:0.0:1787784887.720493:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+59:32851:x1874627658296576:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784887.720973:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+59:32851:x1874627658296576:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 487us (4009us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784887.720982:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1494 00000100:00100000:2.0:1787784887.721835:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784887.724232:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.724277:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.724294:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.724308:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.724438:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.724447:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.724455:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.724463:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.724468:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.724477:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.724493:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.724497:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:0.0:1787784887.724814:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658296832 00000100:00100000:0.0:1787784887.724827:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658296832 00000100:00100000:0.0:1787784887.724914:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 319 00000100:00100000:0.0:1787784887.724926:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+171:32847:x1874627658296832:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.725185:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+172:32847:x1874627658296832:12345-192.168.203.43@tcp:35:cat.0 Request processed in 260us (3352us total) trans 4294968323 rc 0/0 00000100:00100000:0.0:1787784887.725194:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 319 00010000:00010000:1.0:1787784887.725968:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.725980:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7c86 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726007:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726014:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726021:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726026:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.726030:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726038:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726052:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.726055:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.726058:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726063:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7c78 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726073:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726079:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726085:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726090:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.726092:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726098:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726104:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.726106:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.726109:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726115:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7c7f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726187:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7c71 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18db expref: 171 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726199:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.726202:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.726206:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7c71 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18db expref: 171 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.726216:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8800, rc 0) 00000100:00100000:1.0:1787784887.726261:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+170:32830:x1874627658296320:12345-192.168.203.43@tcp:101:cat.0 Request processed in 9171us (9325us total) trans 4294968322 rc 301/301 00000100:00100000:1.0:1787784887.726273:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1493 00000100:00100000:2.0:1787784887.727461:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.730008:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.730644:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658296960 00000100:00100000:2.0:1787784887.730658:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658296960 00000100:00100000:2.0:1787784887.730718:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658297216 00000100:00100000:2.0:1787784887.730721:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658297216 00000100:00100000:2.0:1787784887.730748:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1495 00000100:00100000:2.0:1787784887.730758:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+59:32851:x1874627658296960:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784887.730785:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784887.730813:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110600/0xb0ce63137acc7c94 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18e2 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.730919:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc7c9b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.730938:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc7c9b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.730951:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc7c9b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784887.731000:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784887.731024:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc7c9b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.731057:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110600/0xb0ce63137acc7c94 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18e2 expref: 61 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.731079:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc7c9b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18e2 expref: 60 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.731089:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:2.0:1787784887.731093:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1787784887.731097:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111200, rc 0) 00000100:00100000:2.0:1787784887.731203:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+60:32851:x1874627658296960:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 446us (3758us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784887.731211:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1495 00000100:00100000:3.0:1787784887.731847:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1496 00000100:00100000:3.0:1787784887.731881:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+170:32841:x1874627658297216:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784887.731906:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784887.731923:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7ca2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18e9 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.731996:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732009:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732018:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732037:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732044:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732049:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732222:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732234:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732242:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784887.732251:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1787784887.732377:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732385:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732392:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732398:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.732404:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732416:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732429:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:3.0:1787784887.732432:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784887.732436:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732443:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7cb7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732460:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732466:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732471:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732475:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.732478:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732484:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.732491:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.732493:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.732496:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735630:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7ca9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735658:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735666:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735683:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735689:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784887.735694:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735702:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735721:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff9895198b5200) 00010000:00010000:3.0:1787784887.735724:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784887.735729:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735735:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7c00/0xb0ce63137acc7cb0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735815:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7ca2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18e9 expref: 172 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735833:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:3.0:1787784887.735835:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787784887.735839:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6400/0xb0ce63137acc7ca2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18e9 expref: 172 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784887.735850:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784887.735957:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+171:32841:x1874627658297216:12345-192.168.203.43@tcp:101:cat.0 Request processed in 4097us (5958us total) trans 4294968324 rc 301/301 00000100:00100000:3.0:1787784887.735967:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1496 00000100:00100000:2.0:1787784887.736172:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.736530:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658297472 00000100:00100000:0.0:1787784887.736541:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658297472 00000100:00100000:0.0:1787784887.736597:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00100000:0.0:1787784887.736607:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+171:32830:x1874627658297472:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.736889:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+172:32830:x1874627658297472:12345-192.168.203.43@tcp:35:cat.0 Request processed in 282us (745us total) trans 4294968325 rc 0/0 00000100:00100000:0.0:1787784887.736896:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 320 00000100:00100000:2.0:1787784887.748522:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.748687:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658297728 00000100:00100000:1.0:1787784887.748701:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658297728 00000100:00100000:1.0:1787784887.748777:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1497 00000100:00100000:1.0:1787784887.748790:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+172:32838:x1874627658297728:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.748823:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.748851:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7cc5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18f0 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.748966:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.748983:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.748996:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749013:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749021:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749028:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749228:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749245:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749252:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.749278:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.749398:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749407:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749414:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749421:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.749425:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749434:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749448:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.749452:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.749458:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749465:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7cda lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749488:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.749495:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784887.750948:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.751223:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658297984 00000100:00100000:0.0:1787784887.751234:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658297984 00000100:00100000:0.0:1787784887.751308:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 321 00000100:00100000:0.0:1787784887.751320:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+175:32841:x1874627658297984:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.751691:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+176:32841:x1874627658297984:12345-192.168.203.43@tcp:35:cat.0 Request processed in 371us (752us total) trans 4294968327 rc 0/0 00000100:00100000:0.0:1787784887.751701:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 321 00010000:00010000:1.0:1787784887.752617:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.752625:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.752631:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.752641:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.752660:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.752663:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.752669:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753079:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7ccc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753102:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753108:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753115:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753122:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.753125:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753131:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753139:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.753142:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.753175:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753182:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cd3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753253:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7cc5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f0 expref: 175 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753356:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.753359:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.753363:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7cc5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f0 expref: 175 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.753374:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8800, rc 0) 00000100:00100000:1.0:1787784887.753495:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+174:32838:x1874627658297728:12345-192.168.203.43@tcp:101:cat.0 Request processed in 4708us (4998us total) trans 4294968326 rc 301/301 00000100:00100000:1.0:1787784887.753513:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1497 00000100:00100000:2.0:1787784887.757554:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.758823:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658298112 00000100:00100000:0.0:1787784887.758841:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658298112 00000100:00100000:0.0:1787784887.758941:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 414 00000100:00100000:0.0:1787784887.758952:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527deb050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658298112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784887.758975:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784887.759008:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18f7 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784887.759049:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a8f6eb7, blocks: 0x1 00010000:00010000:0.0:1787784887.759075:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1787784887.759080:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784887.759091:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.759112:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 12 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784887.759128:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:0.0:1787784887.759221:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527deb050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658298112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 271us (1683us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784887.759230:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 414 00000100:00100000:3.0:1787784887.777944:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.778041:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658298368 00000100:00100000:0.0:1787784887.778054:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658298368 00000100:00100000:0.0:1787784887.778124:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 322 00000100:00100000:0.0:1787784887.778136:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d0c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+174:32838:x1874627658298368:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:0.0:1787784887.778484:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d0c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+175:32838:x1874627658298368:12345-192.168.203.43@tcp:35:cat.0 Request processed in 349us (553us total) trans 4294968328 rc 0/0 00000100:00100000:0.0:1787784887.778492:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 322 00000100:00100000:2.0:1787784887.780384:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.780494:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658298624 00000100:00100000:1.0:1787784887.780514:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658298624 00000100:00100000:1.0:1787784887.780582:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1498 00000100:00100000:1.0:1787784887.780593:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+175:32849:x1874627658298624:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.780615:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.780641:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7cef lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b18fe expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780738:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780756:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780769:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780786:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780793:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780799:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780982:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.780994:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781002:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.781013:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.781119:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781127:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781134:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781140:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.781197:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781208:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781221:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.781225:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.781230:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781238:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d04 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781258:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781263:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781273:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781279:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.781282:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781289:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781296:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.781299:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.781301:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781307:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7cf6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781316:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781323:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781329:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781334:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.781337:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781347:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781354:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.781357:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.781360:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781365:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7cfd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781416:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7cef lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18fe expref: 177 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781428:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.781430:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.781434:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7cef lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18fe expref: 177 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.781444:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8800, rc 0) 00000100:00100000:1.0:1787784887.781536:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+176:32849:x1874627658298624:12345-192.168.203.43@tcp:101:cat.0 Request processed in 943us (1162us total) trans 4294968329 rc 301/301 00000100:00100000:1.0:1787784887.781545:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1498 00000100:00100000:2.0:1787784887.789105:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784887.789210:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658298880 00000100:00100000:3.0:1787784887.789222:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658298880 00000100:00100000:3.0:1787784887.789296:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1499 00000100:00100000:3.0:1787784887.789308:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+60:32851:x1874627658298880:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784887.789467:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+60:32851:x1874627658298880:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 160us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784887.789475:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1499 00000100:00100000:2.0:1787784887.794110:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.794268:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658299264 00000100:00100000:1.0:1787784887.794280:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658299264 00000100:00100000:1.0:1787784887.794357:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1500 00000100:00100000:1.0:1787784887.794368:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+176:32846:x1874627658299264:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.794405:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.794431:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7d12 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b190c expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794686:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794703:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794716:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794733:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794740:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794746:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794920:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794931:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.794939:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.794950:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.795069:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795077:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795083:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795090:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.795095:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795103:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795117:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.795120:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.795126:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795132:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d27 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795200:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795208:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795213:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795223:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.795227:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795232:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795240:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.795242:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.795245:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795251:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d19 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795260:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795267:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795273:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795278:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.795281:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795287:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795298:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.795301:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.795304:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795310:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d20 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795358:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7d12 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b190c expref: 178 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795370:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.795373:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.795376:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7d12 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b190c expref: 178 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.795386:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8800, rc 0) 00000100:00100000:1.0:1787784887.796042:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+177:32846:x1874627658299264:12345-192.168.203.43@tcp:101:cat.0 Request processed in 1672us (1943us total) trans 4294968330 rc 301/301 00000100:00100000:1.0:1787784887.796054:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1500 00000100:00100000:0.0:1787784887.800045:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.800089:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.800279:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658299520 00000100:00100000:2.0:1787784887.800293:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658299520 00000100:00100000:2.0:1787784887.800364:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 323 00000100:00100000:2.0:1787784887.800377:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+177:32849:x1874627658299520:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.800719:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+178:32849:x1874627658299520:12345-192.168.203.43@tcp:35:cat.0 Request processed in 343us (627us total) trans 4294968331 rc 0/0 00000100:00100000:2.0:1787784887.800728:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 323 00000100:00100000:1.0:1787784887.801369:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658299008 00000100:00100000:1.0:1787784887.801379:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658299008 00000100:00100000:1.0:1787784887.801432:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:1.0:1787784887.801443:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:32851:x1874627658299008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784887.801456:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.801480:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1905 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.801535:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.801547:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784887.801552:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784887.801559:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 13 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.801602:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 13 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.801617:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 13 pid: 13825 timeout: 778 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784887.801633:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc0b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668448896:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784887.801725:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72180 going to sleep for 11 seconds 00000100:00100000:0.0:1787784887.825093:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.825236:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc0b40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668448896:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784887.825284:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 13 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.825311:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8e00, rc 0) 00000100:00100000:1.0:1787784887.825395:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:32851:x1874627658299008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 23947us (25358us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784887.825406:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 415 00000100:00100000:2.0:1787784887.825865:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658299648 00000100:00100000:2.0:1787784887.825883:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658299648 00010000:00010000:2.0:1787784887.825926:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 13 pid: 13825 timeout: 778 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.825957:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 435 00000100:00100000:2.0:1787784887.825967:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c96050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658299648:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784887.825985:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784887.826013:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 13 pid: 13825 timeout: 778 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.826029:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 13 pid: 13825 timeout: 778 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.826057:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7ce8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18f7 expref: 13 pid: 13825 timeout: 778 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.826073:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784887.826078:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784887.826086:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 12 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.826097:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 12 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784887.826135:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 12 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784887.826280:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b12d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668449024:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784887.826410:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff989503f6f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784887.839845:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.839956:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784887.840254:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658300288 00000100:00100000:1.0:1787784887.840267:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658300288 00000100:00100000:1.0:1787784887.840342:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1501 00000100:00100000:1.0:1787784887.840353:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+178:32843:x1874627658300288:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784887.840385:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784887.840424:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7d3c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1913 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840546:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840564:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840576:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840593:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840600:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840606:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840762:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840773:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840785:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784887.840795:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1787784887.840921:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840930:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840937:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840944:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.840949:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840957:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840971:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784887.840975:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784887.840980:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.840988:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9400/0xb0ce63137acc7d51 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841009:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841019:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841025:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841031:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.841034:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841040:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841048:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.841051:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.841054:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841059:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9600/0xb0ce63137acc7d43 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841068:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841073:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841080:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841085:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784887.841089:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841094:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841100:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff98951b73b200) 00010000:00010000:1.0:1787784887.841103:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784887.841106:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841112:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9e00/0xb0ce63137acc7d4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841212:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7d3c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1913 expref: 180 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841224:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff989520108700) 00010000:00010000:1.0:1787784887.841226:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787784887.841230:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7d3c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1913 expref: 180 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784887.841241:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8800, rc 0) 00000100:00100000:1.0:1787784887.841319:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+179:32843:x1874627658300288:12345-192.168.203.43@tcp:101:cat.0 Request processed in 967us (1489us total) trans 4294968332 rc 301/301 00000100:00100000:1.0:1787784887.841328:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1501 00000100:00100000:2.0:1787784887.841986:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b12d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668449024:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784887.842018:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784887.842057:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c96050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658299648:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 16091us (16980us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784887.842066:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 435 00000100:00100000:2.0:1787784887.842137:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658300032 00000100:00100000:2.0:1787784887.842167:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658300032 00000100:00100000:2.0:1787784887.842210:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:2.0:1787784887.842219:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+179:32846:x1874627658300032:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.842618:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+180:32846:x1874627658300032:12345-192.168.203.43@tcp:35:cat.0 Request processed in 397us (2660us total) trans 4294968333 rc 0/0 00000100:00100000:2.0:1787784887.842626:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 324 00000100:00100000:1.0:1787784887.855544:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784887.855727:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658299776 00000100:00100000:0.0:1787784887.855739:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658299776 00002000:00100000:0.0:1787784887.855808:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1f:0x0] (0->157) req@ffff989401cc0b40 x1874627658299776/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:13/0 lens 648/0 e 0 to 0 dl 1787784898 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784887.855821:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cc0b40 with x1874627658299776 ext(0->157) 00002000:00010000:0.0:1787784887.855827:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401cc0b40 00000100:00100000:0.0:1787784887.855874:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 123 00000100:00100000:0.0:1787784887.855886:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658299776:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1787784887.857286:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658299776:12345-192.168.203.43@tcp:4:lhsmtool_posix.0 Request processed in 1391us (1747us total) trans 4294967387 rc 0/0 00000100:00100000:0.0:1787784887.857300:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 123 00002000:00100000:0.0:1787784887.857311:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1f:0x0] (0->157) req@ffff989401cc0b40 x1874627658299776/t4294967387(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:13/0 lens 648/448 e 0 to 0 dl 1787784898 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784887.857321:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cc0b40 with x1874627658299776 ext(0->157) 00002000:00010000:0.0:1787784887.857326:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401cc0b40 00000100:00100000:3.0:1787784887.868927:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.869053:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658300544 00000100:00100000:2.0:1787784887.869072:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658300544 00000100:00100000:2.0:1787784887.869206:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:2.0:1787784887.869219:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+180:32843:x1874627658300544:12345-192.168.203.43@tcp:35:cat.0 00000100:00100000:2.0:1787784887.871885:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+181:32843:x1874627658300544:12345-192.168.203.43@tcp:35:cat.0 Request processed in 2642us (2939us total) trans 4294968334 rc 0/0 00000100:00100000:2.0:1787784887.871903:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 325 00000100:00100000:0.0:1787784887.891841:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784887.891987:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658300672 00000100:00100000:2.0:1787784887.891999:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658300672 00000100:00100000:2.0:1787784887.892119:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1502 00000100:00100000:2.0:1787784887.892132:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+60:32851:x1874627658300672:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1787784888.000397:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968328 is committed 00000001:00080000:3.0:1787784888.000467:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968256 is committed 00000001:00080000:3.0:1787784888.000475:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968332 is committed 00000001:00080000:3.0:1787784888.000497:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968334 is committed 00000001:00080000:3.0:1787784888.000515:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968312 is committed 00000004:00080000:3.0:1787784888.000519:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 21 committed 00000001:00080000:3.0:1787784888.000580:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968320 is committed 00000004:00080000:3.0:1787784888.000622:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 16 committed 00000100:00100000:3.0:1787784888.000805:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ad2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668449152:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784888.001125:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784888.004457:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658300672:12345-192.168.203.43@tcp:44:lhsmtool_posix.0 Request processed in 112298us (112601us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784888.004474:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1502 00000100:00100000:3.0:1787784888.005273:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668449280:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1787784888.005353:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784888.023186:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668449152 00000100:00100000:0.0:1787784888.023206:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668449152 00000100:00100000:0.0:1787784888.023290:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668449280 00000100:00100000:0.0:1787784888.023295:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668449280 00000100:00100000:0.0:1787784888.023325:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:0.0:1787784888.023336:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668449152:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1787784888.023376:0:6669:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:28 count 1 00010000:00010000:0.0:1787784888.023408:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784888.023595:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:3.0:1787784888.023606:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668449280:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1787784888.023818:0:10688:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1787784888.023822:0:10688:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb mtime from disk: 0 -> 1787784887 00002000:00010000:3.0:1787784888.023824:0:10688:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1787784888.023826:0:10688:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb ctime from disk: 1787784887 -> 1787784887 00002000:00010000:3.0:1787784888.023828:0:10688:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1787784888.023938:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668449280:12345-0@lo:2:osp-syn-1-0.0 Request processed in 334us (18584us total) trans 4294967388 rc 0/0 00000100:00100000:3.0:1787784888.023945:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00080000:1.0:1787784888.024173:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784888.024179:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787784888.024254:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.024267:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 16 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.024277:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=2s v=(0 0) 00010000:00010000:0.0:1787784888.024282:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.024287:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 5/0,1 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.024333:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 16 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.024350:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 16 pid: 6671 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.024369:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668449408:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784888.024693:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.024719:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658300800 00000100:00100000:1.0:1787784888.024724:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658300800 00000100:00100000:1.0:1787784888.024771:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 418 00000100:00100000:1.0:1787784888.024779:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2354:x1874627658300800:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 00000004:00080000:3.0:1787784888.025848:0:3279:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401cc2d00/2, rc 0, transno 4294967388 00000100:00100000:3.0:1787784888.026047:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668449280:0@lo:2:osp-syn-1-0.0 00010000:00010000:1.0:1787784888.032766:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 10 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.032794:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 10 pid: 6669 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784888.032814:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668449536:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784888.032924:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051de40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784888.083908:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.086158:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668449408:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784888.089284:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658300928 00000100:00100000:0.0:1787784888.089303:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658300928 00010000:00010000:0.0:1787784888.089385:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 17 pid: 6671 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.089618:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00100000:0.0:1787784888.089633:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658300928:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784888.089653:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784888.089660:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 17 pid: 6671 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.089678:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 17 pid: 6671 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.089709:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc77cb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17ca expref: 17 pid: 6671 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.089732:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784888.091292:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658300928:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1650us (7386us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.091307:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00080000:2.0:1787784888.091479:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784888.096970:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.097016:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.097051:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658301184 00000100:00100000:2.0:1787784888.097064:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658301184 00000100:00100000:2.0:1787784888.097122:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 613 00000100:00100000:2.0:1787784888.097133:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658301184:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784888.098892:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658301056 00000100:00100000:0.0:1787784888.098906:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658301056 00010000:00010000:0.0:1787784888.098969:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 11 pid: 6669 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.099012:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 437 00000100:00100000:0.0:1787784888.099022:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658301056:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784888.099039:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784888.099045:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 11 pid: 6669 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.099067:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 11 pid: 6669 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.099100:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.099106:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.099113:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.099125:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.099179:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:2.0:1787784888.099622:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784888.099660:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658301184:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 2526us (2640us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784888.099672:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 613 00010000:00010000:1.0:1787784888.100287:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc77a8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b17bc expref: 11 pid: 6669 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784888.100308:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668449536:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784888.100339:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.100351:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.100360:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.100378:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.100391:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.100415:0:6669:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787784888.101819:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658301056:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2784us (4852us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.101836:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 437 00000100:00100000:1.0:1787784888.102771:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668449152:12345-0@lo:6:osp-syn-0-0.0 Request processed in 79432us (101641us total) trans 4294967375 rc 0/0 00000100:00100000:1.0:1787784888.102783:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00010000:00010000:1.0:1787784888.102818:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784888.102829:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.102844:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784888.102848:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.102860:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.106801:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.106815:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b40387f/0xb0ce63137acc7d5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.106850:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784888.106853:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784888.106857:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787784888.106938:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9894021ad2c0/2, rc 0, transno 4294967375 00000100:00100000:1.0:1787784888.106948:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ad2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668449152:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787784888.107945:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784888.108101:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658301312 00000100:00100000:0.0:1787784888.108111:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658301312 00000100:00100000:0.0:1787784888.108209:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 457 00000100:00100000:0.0:1787784888.108230:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658301312:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784888.108310:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658301312:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 92us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.108318:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 457 00000100:00080000:2.0:1787784888.108841:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00080000:3.0:1787784888.126242:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967387 is committed 00000001:00080000:3.0:1787784888.130605:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967388 is committed 00000100:00100000:1.0:1787784888.130838:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2354:x1874627658300800:12345-192.168.203.43@tcp:16:lhsmtool_posix.0 Request processed in 106051us (106138us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.130853:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 418 00000100:00100000:2.0:1787784888.135760:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784888.136002:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658301568 00000100:00100000:0.0:1787784888.136016:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658301568 00000100:00100000:0.0:1787784888.136091:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1503 00000100:00100000:0.0:1787784888.136104:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658301568:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1787784888.136215:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784888.136255:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc7c9b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18e2 expref: 53 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.136280:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:0.0:1787784888.136286:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784888.136328:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.136344:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.136355:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784888.136633:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bfe7997d reply state 00000000adc59709 transno 4294968335 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.136645:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784888.136772:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:32851:x1874627658301568:12345-192.168.203.43@tcp:36:lhsmtool_posix.0 Request processed in 669us (1038us total) trans 4294968335 rc 0/0 00000100:00100000:0.0:1787784888.136780:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1503 00000100:00100000:2.0:1787784888.141083:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.141126:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658301696 00000100:00100000:2.0:1787784888.141138:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658301696 00000100:00100000:2.0:1787784888.141230:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 419 00000100:00100000:2.0:1787784888.141241:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2355:x1874627658301696:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1787784888.141575:0:13825:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:2.0:1787784888.141579:0:13825:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb mtime from disk: 1787784887 -> 1787784884 00002000:00010000:2.0:1787784888.141582:0:13825:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb atime from disk: 0 -> 1787784884 00002000:00010000:2.0:1787784888.141584:0:13825:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb ctime from disk: 1787784887 -> 1787784886 00002000:00010000:2.0:1787784888.141586:0:13825:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1787784888.141656:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658301696:12345-192.168.203.43@tcp:2:lhsmtool_posix.0 Request processed in 416us (585us total) trans 4294967389 rc 0/0 00000100:00100000:2.0:1787784888.141664:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 419 00010000:00010000:3.0:1787784888.142096:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.142109:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.142118:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.142125:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784888.142131:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.142140:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.142182:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:3.0:1787784888.142185:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784888.142191:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.142198:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd800/0xb0ce63137acc7d66 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784888.149757:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784888.149834:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658301952 00000100:00100000:0.0:1787784888.149847:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658301952 00000100:00100000:0.0:1787784888.149931:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1504 00000100:00100000:0.0:1787784888.149942:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+53:32851:x1874627658301952:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1787784888.150031:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.150054:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 53 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.150066:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.150110:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 53 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.150127:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 53 pid: 5811 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.150142:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668449664:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784888.150287:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d540 going to sleep for 11 seconds 00000100:00100000:2.0:1787784888.156268:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.156371:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658302080 00000100:00100000:1.0:1787784888.156393:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658302080 00010000:00010000:1.0:1787784888.156447:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 54 pid: 5811 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784888.156506:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 438 00000100:00100000:1.0:1787784888.156517:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658302080:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784888.156535:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784888.156544:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 54 pid: 5811 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.156559:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 54 pid: 5811 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.156587:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:1.0:1787784888.156594:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.156610:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.156618:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.156642:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784888.156703:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658302080:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 187us (449us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.156711:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 438 00010000:00010000:0.0:1787784888.157173:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad000/0xb0ce63137acc7bad lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18aa expref: 53 pid: 5811 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.157197:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668449664:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784888.157223:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157233:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157238:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157275:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157283:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157292:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157297:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784888.157302:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157325:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157337:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:0.0:1787784888.157340:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787784888.157344:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.157351:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894022fd200/0xb0ce63137acc7d6d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.157416:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32851:x1874627658301952:12345-192.168.203.43@tcp:55:lhsmtool_posix.0 Request processed in 7476us (7683us total) trans 0 rc -22/-22 00000100:00100000:0.0:1787784888.157425:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1504 00000100:00100000:3.0:1787784888.161410:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784888.161619:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658302208 00000100:00100000:0.0:1787784888.161631:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658302208 00000100:00100000:0.0:1787784888.161708:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:0.0:1787784888.161719:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658302208:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784888.161787:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.161812:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 12 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.161822:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1f:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784888.161827:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784888.161832:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.161889:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 12 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.161907:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 12 pid: 6670 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.161924:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668449792:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784888.162030:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787784888.173793:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668449792:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784888.173836:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.182305:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784888.182913:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658302336 00000100:00100000:0.0:1787784888.182927:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658302336 00010000:00010000:0.0:1787784888.182984:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 13 pid: 6670 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.183042:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 439 00000100:00100000:0.0:1787784888.183054:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658302336:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784888.183074:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784888.183103:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 13 pid: 6670 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.183120:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 13 pid: 6670 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.183169:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7d35 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1905 expref: 13 pid: 6670 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.183192:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784888.183197:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784888.183204:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.183216:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.183259:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784888.183326:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658302336:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 274us (1036us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.183335:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 439 00010000:00010000:2.0:1787784888.184018:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184032:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184066:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184075:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184085:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184094:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9894022fd000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784888.184099:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184110:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184124:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.184133:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7d74 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.184245:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658302208:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 22525us (22860us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784888.184256:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 420 00000100:00100000:0.0:1787784888.190776:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.190951:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658302592 00000100:00100000:2.0:1787784888.190964:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658302592 00000100:00100000:2.0:1787784888.191030:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1505 00000100:00100000:2.0:1787784888.191042:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+52:32851:x1874627658302592:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784888.191171:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.191196:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.191211:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.191535:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.191548:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.191556:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.191600:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 53 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.191622:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 53 pid: 5812 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.191639:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539131680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668449920:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784888.191982:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784888.199617:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539131680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668449920:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784888.199677:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784888.204352:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.205199:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658302720 00000100:00100000:2.0:1787784888.205214:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658302720 00010000:00010000:2.0:1787784888.205274:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 54 pid: 5812 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.205374:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:2.0:1787784888.205396:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658302720:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784888.205414:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784888.205422:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 54 pid: 5812 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.205439:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 54 pid: 5812 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.205478:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ac600/0xb0ce63137acc7c01 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b18bf expref: 54 pid: 5812 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.205513:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:2.0:1787784888.205519:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.205536:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.205546:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.205571:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784888.205659:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+53:2357:x1874627658302720:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 266us (1314us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784888.205669:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 440 00010000:00010000:1.0:1787784888.206307:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.206318:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787784888.206568:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968336 00000001:00080000:1.0:1787784888.209964:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968337 00000001:00080000:1.0:1787784888.210094:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968338 00000100:00100000:1.0:1787784888.210240:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668450048:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787784888.210329:0:20438:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784888.210379:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508835900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784888.212587:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668450048 00000100:00100000:3.0:1787784888.212608:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668450048 00000100:00100000:3.0:1787784888.212828:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 73 00000100:00100000:3.0:1787784888.212843:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:20438:x1874627668450048:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787784888.213023:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:20438:x1874627668450048:12345-0@lo:1000:mdt00_004.0 Request processed in 184us (2694us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784888.213031:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 73 00000100:00100000:1.0:1787784888.213119:0:20438:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967388 gen 2 00000004:00080000:1.0:1787784888.213127:0:20438:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401cc2d00, transno 4294967388 00000100:00100000:1.0:1787784888.213168:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668450048:0@lo:1000:mdt00_004.0 00000001:00080000:1.0:1787784888.213696:0:20438:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968339 00000100:00100000:1.0:1787784888.213789:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668450176:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787784888.213835:0:20438:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784888.214672:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989508835840 going to sleep for 11 seconds 00000100:00100000:0.0:1787784888.215877:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668450176 00000100:00100000:0.0:1787784888.215892:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668450176 00000100:00100000:0.0:1787784888.215951:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 74 00000100:00100000:0.0:1787784888.215962:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:20438:x1874627668450176:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787784888.217414:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20438:x1874627668450176:12345-0@lo:1000:mdt00_004.0 Request processed in 1443us (3568us total) trans 4294967390 rc 0/0 00000100:00100000:1.0:1787784888.217428:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1787784888.217842:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:20438:1874627668450176:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787784888.217967:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.217979:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.217988:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.217995:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784888.218000:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218008:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218027:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520131500) 00010000:00010000:2.0:1787784888.218030:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784888.218036:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218042:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7d82 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218079:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218084:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x50210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218090:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218095:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989503c16a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784888.218098:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40210400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218103:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x40218400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218110:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784888.218112:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787784888.218120:0:20438:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1810 expref: 107 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218133:0:20438:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1810 expref: 107 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.218142:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1810 expref: 107 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.221025:0:20438:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403641600/0xb0ce63137acc78ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1810 expref: 107 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.221046:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668450304:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784888.221407:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787784888.226330:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.226623:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668450304:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784888.226662:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.226676:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989503c16a00/0xb0ce63137acc788f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT flags: 0x44a19400000020 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.226720:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.226737:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.226814:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.226936:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.226956:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 107 pid: 20438 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.226979:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668450432:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784888.227070:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504d9cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784888.229105:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658302848 00000100:00100000:3.0:1787784888.229122:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658302848 00000100:00100000:2.0:1787784888.229420:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 326 00000100:00100000:2.0:1787784888.229432:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32821:x1874627658302848:12345-192.168.203.43@tcp:49:cat.0 00000100:00100000:2.0:1787784888.229627:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:32821:x1874627658302848:12345-192.168.203.43@tcp:49:cat.0 Request processed in 196us (3311us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784888.229636:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 326 00000100:00100000:2.0:1787784888.230129:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668450432:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784888.230189:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.241356:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784888.254338:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254395:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254425:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254459:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254489:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254505:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254516:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254533:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254547:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254559:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254575:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254594:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254635:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784888.254688:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658302976 00000100:00100000:1.0:1787784888.254700:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658302976 00000100:00100000:1.0:1787784888.254765:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658303104 00000100:00100000:1.0:1787784888.254769:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658303104 00000100:00100000:1.0:1787784888.254783:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658303232 00000100:00100000:1.0:1787784888.254786:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658303232 00000100:00100000:1.0:1787784888.254796:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658303360 00000100:00100000:1.0:1787784888.254799:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658303360 00000100:00100000:1.0:1787784888.254808:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658303488 00000100:00100000:1.0:1787784888.254811:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658303488 00000100:00100000:1.0:1787784888.254820:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658303616 00000100:00100000:1.0:1787784888.254823:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658303616 00000100:00100000:1.0:1787784888.254831:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658303744 00000100:00100000:1.0:1787784888.254833:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658303744 00000100:00100000:1.0:1787784888.254841:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658303872 00000100:00100000:1.0:1787784888.254843:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658303872 00000100:00100000:1.0:1787784888.254851:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304000 00000100:00100000:1.0:1787784888.254853:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304000 00000100:00100000:1.0:1787784888.254877:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304128 00000100:00100000:1.0:1787784888.254879:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304128 00000100:00100000:1.0:1787784888.254890:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304384 00000100:00100000:1.0:1787784888.254892:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304384 00000100:00100000:1.0:1787784888.254899:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304512 00000100:00100000:1.0:1787784888.254901:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304512 00000100:00100000:1.0:1787784888.254916:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 421 00000100:00100000:1.0:1787784888.254927:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32821:x1874627658302976:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.254950:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.254979:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a7fea1db/0xb0ce63137acc7d90 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b191a expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784888.255014:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1f:0x0] initial LVB size: 158, mtime: 0x6a8f6eb4, atime: 0x6a8f6eb4, ctime: 0x6a8f6eb6, blocks: 0xb 00010000:00010000:1.0:1787784888.255042:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=679s v=(0 0) 00010000:00010000:1.0:1787784888.255047:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255059:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a7fea1db/0xb0ce63137acc7d90 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b191a expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255079:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a7fea1db/0xb0ce63137acc7d90 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b191a expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255095:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9400, rc 0) 00000100:00100000:1.0:1787784888.255196:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32821:x1874627658302976:12345-192.168.203.43@tcp:101:cat.0 Request processed in 270us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255203:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 421 00000100:00100000:1.0:1787784888.255225:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 422 00000100:00100000:1.0:1787784888.255233:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32822:x1874627658303104:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255240:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255249:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f282bfe/0xb0ce63137acc7d97 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1921 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255262:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255265:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255270:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004f282bfe/0xb0ce63137acc7d97 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1921 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255281:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004f282bfe/0xb0ce63137acc7d97 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1921 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255290:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9200, rc 0) 00000100:00100000:1.0:1787784888.255310:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32822:x1874627658303104:12345-192.168.203.43@tcp:101:cat.0 Request processed in 78us (925us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255316:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 422 00000100:00100000:1.0:1787784888.255329:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 423 00000100:00100000:1.0:1787784888.255336:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32825:x1874627658303232:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255343:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255351:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f619b339/0xb0ce63137acc7d9e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1928 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255363:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255366:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255373:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f619b339/0xb0ce63137acc7d9e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1928 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255383:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f619b339/0xb0ce63137acc7d9e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1928 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255391:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9a00, rc 0) 00000100:00100000:1.0:1787784888.255409:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32825:x1874627658303232:12345-192.168.203.43@tcp:101:cat.0 Request processed in 74us (984us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255416:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 423 00000100:00100000:1.0:1787784888.255430:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 424 00000100:00100000:1.0:1787784888.255436:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32823:x1874627658303360:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255443:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255450:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c5d155fa/0xb0ce63137acc7da5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b192f expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255463:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255466:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255472:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c5d155fa/0xb0ce63137acc7da5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b192f expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255482:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c5d155fa/0xb0ce63137acc7da5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b192f expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255490:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8c00, rc 0) 00000100:00100000:1.0:1787784888.255509:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32823:x1874627658303360:12345-192.168.203.43@tcp:101:cat.0 Request processed in 73us (1060us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255515:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 424 00000100:00100000:1.0:1787784888.255528:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 425 00000100:00100000:1.0:1787784888.255534:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32819:x1874627658303488:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255540:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255547:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d2f8140c/0xb0ce63137acc7dac lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1936 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255559:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255563:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255568:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d2f8140c/0xb0ce63137acc7dac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1936 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255578:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d2f8140c/0xb0ce63137acc7dac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1936 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255587:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8600, rc 0) 00000100:00100000:1.0:1787784888.255605:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32819:x1874627658303488:12345-192.168.203.43@tcp:101:cat.0 Request processed in 71us (1115us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255611:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 425 00000100:00100000:1.0:1787784888.255625:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 426 00000100:00100000:1.0:1787784888.255630:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32824:x1874627658303616:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255637:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255644:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000050ad89b0/0xb0ce63137acc7db3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b193d expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255662:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255665:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255670:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000050ad89b0/0xb0ce63137acc7db3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b193d expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255680:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000050ad89b0/0xb0ce63137acc7db3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b193d expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255687:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9c00, rc 0) 00000100:00100000:1.0:1787784888.255706:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32824:x1874627658303616:12345-192.168.203.43@tcp:101:cat.0 Request processed in 76us (1200us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255713:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 426 00000100:00100000:1.0:1787784888.255725:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 427 00000100:00100000:1.0:1787784888.255731:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32827:x1874627658303744:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255738:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255745:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002da4a4a4/0xb0ce63137acc7dba lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1944 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255756:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255759:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255765:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002da4a4a4/0xb0ce63137acc7dba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1944 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255775:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002da4a4a4/0xb0ce63137acc7dba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1944 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255784:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8a00, rc 0) 00000100:00100000:1.0:1787784888.255802:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32827:x1874627658303744:12345-192.168.203.43@tcp:101:cat.0 Request processed in 72us (1285us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255808:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 427 00000100:00100000:1.0:1787784888.255820:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 428 00000100:00100000:1.0:1787784888.255827:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32829:x1874627658303872:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255833:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255840:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005ed7a2ad/0xb0ce63137acc7dc1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b194b expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255854:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255857:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255862:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005ed7a2ad/0xb0ce63137acc7dc1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b194b expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255885:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005ed7a2ad/0xb0ce63137acc7dc1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b194b expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255892:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9800, rc 0) 00000100:00100000:1.0:1787784888.255911:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32829:x1874627658303872:12345-192.168.203.43@tcp:101:cat.0 Request processed in 85us (1378us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.255916:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 428 00000100:00100000:1.0:1787784888.255929:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 429 00000100:00100000:1.0:1787784888.255935:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32835:x1874627658304000:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.255943:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.255950:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002e85cb9c/0xb0ce63137acc7dc8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 10 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1952 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255961:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255964:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.255969:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002e85cb9c/0xb0ce63137acc7dc8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1952 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255978:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002e85cb9c/0xb0ce63137acc7dc8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1952 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.255986:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8200, rc 0) 00000100:00100000:1.0:1787784888.256000:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32835:x1874627658304000:12345-192.168.203.43@tcp:101:cat.0 Request processed in 66us (1453us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.256007:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 429 00000100:00100000:1.0:1787784888.256019:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:1.0:1787784888.256025:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32836:x1874627658304128:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.256031:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.256038:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7dcf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 11 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1959 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256050:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.256053:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.256059:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7dcf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 11 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1959 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256074:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e69029af/0xb0ce63137acc7dcf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 11 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1959 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256083:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8e00, rc 0) 00000100:00100000:1.0:1787784888.256096:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32836:x1874627658304128:12345-192.168.203.43@tcp:101:cat.0 Request processed in 71us (1537us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.256102:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 430 00000100:00100000:1.0:1787784888.256114:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 431 00000100:00100000:1.0:1787784888.256121:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32828:x1874627658304384:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.256127:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.256134:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe2aad23/0xb0ce63137acc7dd6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 12 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1960 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256174:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.256178:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.256184:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe2aad23/0xb0ce63137acc7dd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 12 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1960 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256195:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fe2aad23/0xb0ce63137acc7dd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 12 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1960 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256204:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9600, rc 0) 00000100:00100000:1.0:1787784888.256220:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32828:x1874627658304384:12345-192.168.203.43@tcp:101:cat.0 Request processed in 100us (1625us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.256230:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 431 00000100:00100000:1.0:1787784888.256244:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 432 00000100:00100000:1.0:1787784888.256251:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+27:32826:x1874627658304512:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:1.0:1787784888.256257:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784888.256264:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa483fdf/0xb0ce63137acc7ddd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 13 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1967 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256275:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.256279:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784888.256284:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa483fdf/0xb0ce63137acc7ddd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 13 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1967 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256293:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aa483fdf/0xb0ce63137acc7ddd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 13 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1967 expref: 28 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.256302:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd9e00, rc 0) 00000100:00100000:1.0:1787784888.256317:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+28:32826:x1874627658304512:12345-192.168.203.43@tcp:101:cat.0 Request processed in 67us (1683us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784888.256322:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 432 00000100:00100000:2.0:1787784888.257258:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304256 00000100:00100000:2.0:1787784888.257268:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304256 00010000:00010000:2.0:1787784888.257307:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 108 pid: 20438 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.257341:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:2.0:1787784888.257349:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:2356:x1874627658304256:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784888.257362:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784888.257368:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 108 pid: 20438 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.257394:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 108 pid: 20438 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.257419:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc7922 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1825 expref: 108 pid: 20438 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.257438:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:2.0:1787784888.257444:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.257466:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.257473:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.257505:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784888.257534:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2356:x1874627658304256:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 185us (2957us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784888.257542:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:2.0:1787784888.259584:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.259617:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784888.259755:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304640 00000100:00100000:3.0:1787784888.259766:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304640 00000100:00100000:3.0:1787784888.259826:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304768 00000100:00100000:3.0:1787784888.259829:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304768 00000100:00100000:3.0:1787784888.259851:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 433 00000100:00100000:3.0:1787784888.259876:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+29:32832:x1874627658304640:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784888.259897:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784888.259920:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc7de4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 14 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b196e expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.259952:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784888.259958:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784888.259969:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc7de4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 14 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b196e expref: 30 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.259987:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005044093d/0xb0ce63137acc7de4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 14 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b196e expref: 30 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.260000:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6400, rc 0) 00000100:00100000:3.0:1787784888.260055:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+30:32832:x1874627658304640:12345-192.168.203.43@tcp:101:cat.0 Request processed in 195us (484us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784888.260063:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 433 00000100:00100000:3.0:1787784888.260083:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 434 00000100:00100000:3.0:1787784888.260090:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+29:32844:x1874627658304768:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:3.0:1787784888.260097:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784888.260105:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc7deb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 15 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1975 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.260119:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784888.260122:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784888.260128:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc7deb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 15 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1975 expref: 30 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.260138:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000059ecaec7/0xb0ce63137acc7deb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 15 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1975 expref: 30 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784888.260173:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7c00, rc 0) 00000100:00100000:3.0:1787784888.260196:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+30:32844:x1874627658304768:12345-192.168.203.43@tcp:101:cat.0 Request processed in 106us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784888.260203:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 434 00000100:00100000:2.0:1787784888.260612:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.260644:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658305280 00000100:00100000:2.0:1787784888.260648:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658305280 00000100:00100000:2.0:1787784888.260686:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 435 00000100:00100000:2.0:1787784888.260693:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+30:32834:x1874627658305280:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:2.0:1787784888.260702:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784888.260712:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a76fee7a/0xb0ce63137acc7df2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 16 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1991 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.260734:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784888.260951:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784888.260960:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a76fee7a/0xb0ce63137acc7df2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 16 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1991 expref: 31 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.260975:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a76fee7a/0xb0ce63137acc7df2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 16 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1991 expref: 31 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784888.260986:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110600, rc 0) 00000100:00100000:2.0:1787784888.261016:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+31:32834:x1874627658305280:12345-192.168.203.43@tcp:101:cat.0 Request processed in 324us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784888.261024:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 435 00010000:00010000:1.0:1787784888.262677:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.262793:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.262813:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.262821:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.262830:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.262836:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784888.262841:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.262849:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.262984:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784888.262989:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784888.262994:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263001:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc7d89 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263324:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263336:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263342:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263351:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784888.263355:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263362:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263375:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff989520131e00) 00010000:00010000:1.0:1787784888.263378:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784888.263381:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.263392:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc7d7b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784888.274056:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784888.274309:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658305408 00000100:00100000:0.0:1787784888.274322:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658305408 00000100:00100000:0.0:1787784888.274397:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00100000:0.0:1787784888.274409:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+31:32831:x1874627658305408:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784888.274432:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784888.274470:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc7df9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 17 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1998 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.274510:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.274515:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.274527:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc7df9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 17 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1998 expref: 32 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.274546:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001ee5c6bd/0xb0ce63137acc7df9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 17 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1998 expref: 32 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.274561:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd800, rc 0) 00000100:00100000:0.0:1787784888.274640:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+32:32831:x1874627658305408:12345-192.168.203.43@tcp:101:cat.0 Request processed in 232us (598us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.274648:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 436 00000100:00100000:1.0:1787784888.276777:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32851:x1874627658302592:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 85722us (86000us total) trans 4294968340 rc 0/0 00000100:00100000:1.0:1787784888.276795:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1505 00000020:00080000:1.0:1787784888.276802:0:20438:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:3.0:1787784888.277304:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784888.277311:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784888.280144:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1787784888.280186:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784888.280554:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.280660:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.280725:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784888.280761:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784888.282915:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658304896 00000100:00100000:0.0:1787784888.282934:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658304896 00000100:00100000:0.0:1787784888.283017:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658305024 00000100:00100000:0.0:1787784888.283021:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658305024 00000100:00100000:0.0:1787784888.283033:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658305536 00000100:00100000:0.0:1787784888.283036:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658305536 00000100:00100000:0.0:1787784888.283046:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658305152 00000100:00100000:0.0:1787784888.283048:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658305152 00000100:00100000:0.0:1787784888.283068:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 437 00000100:00100000:0.0:1787784888.283079:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+35:32848:x1874627658304896:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784888.283103:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784888.283129:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7e00 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 18 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b197c expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283204:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283210:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283222:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7e00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 18 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b197c expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283240:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000a39820/0xb0ce63137acc7e00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 18 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b197c expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283255:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd000, rc 0) 00000100:00100000:0.0:1787784888.283337:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:32848:x1874627658304896:12345-192.168.203.43@tcp:101:cat.0 Request processed in 259us (2790us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.283345:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 437 00000100:00100000:0.0:1787784888.283375:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 438 00000100:00100000:0.0:1787784888.283382:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+35:32842:x1874627658305024:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784888.283389:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784888.283398:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7e07 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 19 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1983 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283411:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283415:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283420:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7e07 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 19 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1983 expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283431:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b40387f/0xb0ce63137acc7e07 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 19 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1983 expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283440:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9894022fd200, rc 0) 00000100:00100000:0.0:1787784888.283461:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:32842:x1874627658305024:12345-192.168.203.43@tcp:101:cat.0 Request processed in 80us (2811us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.283468:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 438 00000100:00100000:0.0:1787784888.283483:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 439 00000100:00100000:0.0:1787784888.283490:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+35:32840:x1874627658305536:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784888.283496:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784888.283504:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001ca94d2c/0xb0ce63137acc7e0e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 20 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b199f expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283519:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283522:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283528:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001ca94d2c/0xb0ce63137acc7e0e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 20 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b199f expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283538:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001ca94d2c/0xb0ce63137acc7e0e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 20 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b199f expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283547:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351a00, rc 0) 00000100:00100000:0.0:1787784888.283566:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:32840:x1874627658305536:12345-192.168.203.43@tcp:101:cat.0 Request processed in 77us (2841us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.283572:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 439 00000100:00100000:0.0:1787784888.283586:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:0.0:1787784888.283593:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+35:32839:x1874627658305152:12345-192.168.203.43@tcp:101:cat.0 00010000:00010000:0.0:1787784888.283599:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784888.283606:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e5451fec/0xb0ce63137acc7e15 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 21 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b198a expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283618:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283621:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784888.283627:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e5451fec/0xb0ce63137acc7e15 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 21 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b198a expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283637:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e5451fec/0xb0ce63137acc7e15 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 21 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b198a expref: 36 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.283645:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989403351800, rc 0) 00000100:00100000:0.0:1787784888.283664:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:32839:x1874627658305152:12345-192.168.203.43@tcp:101:cat.0 Request processed in 73us (2904us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784888.283671:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 440 00000100:00100000:2.0:1787784888.288255:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784888.292641:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658305664 00000100:00100000:3.0:1787784888.292663:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658305664 00002000:00100000:3.0:1787784888.292743:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1f:0x0] (0->4095) req@ffff9895217592c0 x1874627658305664/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:14/0 lens 488/0 e 0 to 0 dl 1787784899 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784888.292756:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895217592c0 with x1874627658305664 ext(0->4095) 00002000:00010000:3.0:1787784888.292762:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9895217592c0 00000100:00100000:3.0:1787784888.292806:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 124 00000100:00100000:3.0:1787784888.292817:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2354:x1874627658305664:12345-192.168.203.43@tcp:3:cat.0 00000100:00100000:3.0:1787784888.293099:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895217592c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2354:x1874627658305664:12345-192.168.203.43@tcp:3:cat.0 Request processed in 283us (4859us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784888.293107:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 124 00002000:00100000:3.0:1787784888.293115:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1f:0x0] (0->4095) req@ffff9895217592c0 x1874627658305664/t0(0) o3->3aa20aaf-dc32-4dd7-b302-1a1d759b54b2@192.168.203.43@tcp:14/0 lens 488/4536 e 0 to 0 dl 1787784899 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784888.293123:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9895217592c0 with x1874627658305664 ext(0->4095) 00002000:00010000:3.0:1787784888.293126:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9895217592c0 00000100:00100000:2.0:1787784888.363090:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784888.363277:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658305920 00000100:00100000:3.0:1787784888.363291:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658305920 00000100:00100000:3.0:1787784888.363366:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 327 00000100:00100000:3.0:1787784888.363378:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+56:32851:x1874627658305920:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1787784888.366999:0:5814:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x2d:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1787784888.367141:0:5814:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968341 00010000:00010000:0.0:1787784888.367267:0:5814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.367289:0:5814:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.367303:0:5814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.367313:0:5814:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.367321:0:5814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.367328:0:5814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784888.367348:0:5814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787784888.367499:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+58:32851:x1874627658305920:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 4117us (4419us total) trans 4294968342 rc 0/0 00000100:00100000:0.0:1787784888.367513:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 327 00000020:00080000:0.0:1787784888.367520:0:5814:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1787784888.370205:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784888.370225:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.370238:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784888.370243:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.370254:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.370277:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff989520108600) 00010000:00010000:1.0:1787784888.370280:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787784888.370286:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.370293:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e1c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5814 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784888.370334:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784888.370337:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784888.370338:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784888.914190:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967375 is committed 00000001:00080000:3.0:1787784888.914216:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967374 is committed 00010000:00010000:1.0:1787784889.265524:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784889.841465:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539250b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668450560:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784889.841582:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784889.841733:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668450560 00000100:00100000:3.0:1787784889.841744:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668450560 00000100:00100000:3.0:1787784889.841816:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1787784889.841828:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668450560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784889.841893:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 93519872 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784889.855621:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668450560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 13780us (14023us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784889.855874:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:2.0:1787784889.859013:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967375 gen 2 00000004:00080000:2.0:1787784889.859031:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9894021ad2c0, transno 4294967375 00000100:00100000:2.0:1787784889.859070:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539250b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668450560:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784890.291504:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784891.314798:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784891.889417:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668450944:0@lo:400:kworker.0 00000100:00100000:3.0:1787784891.889490:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784891.889555:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787784891.889561:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391003c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668450816:0@lo:400:kworker.0 00000100:00100000:3.0:1787784891.889586:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784891.889601:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539103c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668451328:0@lo:400:kworker.0 00000100:00100000:3.0:1787784891.889624:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784891.889790:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668451072:0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.889820:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784891.889862:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787784891.889868:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539101a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668450688:0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.889909:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784891.889934:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539100000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668451200:0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.889957:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784891.889987:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668450944 00000100:00100000:1.0:1787784891.889994:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668450944 00000100:00100000:1.0:1787784891.890043:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668450816 00000100:00100000:1.0:1787784891.890047:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668450816 00000100:00100000:1.0:1787784891.890065:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451200 00000100:00100000:1.0:1787784891.890068:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451200 00000100:00100000:1.0:1787784891.890085:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1787784891.890094:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668450944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.890143:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668450944:12345-0@lo:400:kworker.0 Request processed in 50us (652us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784891.890176:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1787784891.890193:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:1.0:1787784891.890213:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668450816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.890242:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668450816:12345-0@lo:400:kworker.0 Request processed in 32us (656us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784891.890248:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:1.0:1787784891.890259:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:1.0:1787784891.890264:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668451200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.890294:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668451200:12345-0@lo:400:kworker.0 Request processed in 29us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784891.890299:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:1.0:1787784891.890320:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668450688 00000100:00100000:1.0:1787784891.890324:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668450688 00000100:00100000:1.0:1787784891.890347:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:1.0:1787784891.890353:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668450688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.890383:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668450688:12345-0@lo:400:kworker.0 Request processed in 30us (474us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784891.890389:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00080000:1.0:1787784891.890415:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784891.890451:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539101a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668450688:0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.890466:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539100000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668451200:0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.890497:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391003c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668450816:0@lo:400:kworker.0 00000100:00100000:1.0:1787784891.890509:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668450944:0@lo:400:kworker.0 00000100:00100000:3.0:1787784891.890698:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451328 00000100:00100000:3.0:1787784891.890702:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451328 00000100:00100000:3.0:1787784891.890745:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451072 00000100:00100000:3.0:1787784891.890749:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451072 00000100:00100000:3.0:1787784891.890772:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:3.0:1787784891.890779:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668451328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784891.890821:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668451328:12345-0@lo:400:kworker.0 Request processed in 43us (1195us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784891.890827:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:3.0:1787784891.890853:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:3.0:1787784891.890861:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668451072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784891.890896:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668451072:12345-0@lo:400:kworker.0 Request processed in 37us (1075us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784891.890902:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:3.0:1787784891.890935:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668451072:0@lo:400:kworker.0 00000100:00100000:3.0:1787784891.890961:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539103c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668451328:0@lo:400:kworker.0 00000100:00080000:1.0:1787784891.891743:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784891.891749:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784891.902712:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784891.902737:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539103c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668451456:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784891.902809:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784891.902939:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451456 00000100:00100000:3.0:1787784891.902948:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451456 00000100:00100000:3.0:1787784891.903091:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:3.0:1787784891.903102:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668451456:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784891.903166:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784891.903273:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668451456:12345-0@lo:13:osp-pre-1-0.0 Request processed in 173us (463us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784891.903281:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1787784891.903461:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539103c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668451456:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1787784892.341118:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784892.408107:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784892.408122:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784892.409865:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 45 count 3 00000040:00080000:2.0:1787784892.409886:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:1.0:1787784893.010432:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968342 is committed 00000020:00080000:2.0:1787784893.011143:0:22194:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff989508835540 refcount 2 committed 0 result 0 batchid 35 00000020:00080000:2.0:1787784893.011165:0:22194:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953efedc00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff989503f6ff00 00000020:00080000:2.0:1787784893.011169:0:22194:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff98953efed000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff989537042400 00000020:00080000:0.0:1787784893.011284:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 35 committed batchid 34 00000020:00080000:0.0:1787784893.011555:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 34 00000100:00100000:0.0:1787784893.193630:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784893.193668:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784893.193697:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784893.193714:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784893.193730:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784893.193746:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784893.193767:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784893.193792:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306304 00000100:00100000:0.0:1787784893.193801:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306304 00000100:00100000:0.0:1787784893.193861:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306816 00000100:00100000:0.0:1787784893.193864:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306816 00000100:00100000:0.0:1787784893.193880:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306432 00000100:00100000:0.0:1787784893.193882:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306432 00000100:00100000:0.0:1787784893.193895:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306688 00000100:00100000:0.0:1787784893.193897:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306688 00000100:00100000:0.0:1787784893.193912:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:0.0:1787784893.193920:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658306304:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784893.193987:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658306304:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 68us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784893.193994:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:0.0:1787784893.194015:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 461 00000100:00100000:0.0:1787784893.194021:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658306816:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784893.194043:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658306816:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784893.194049:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 461 00000100:00100000:0.0:1787784893.194061:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 462 00000100:00100000:0.0:1787784893.194067:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658306432:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784893.194086:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658306432:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784893.194092:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 462 00000100:00100000:0.0:1787784893.194103:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 463 00000100:00100000:0.0:1787784893.194110:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658306688:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784893.194130:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2356:x1874627658306688:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784893.194136:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 463 00000100:00100000:0.0:1787784893.194187:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306560 00000100:00100000:0.0:1787784893.194191:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306560 00000100:00100000:0.0:1787784893.194222:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306176 00000100:00100000:0.0:1787784893.194226:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306176 00000100:00100000:0.0:1787784893.194247:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 492 00000100:00100000:0.0:1787784893.194253:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2356:x1874627658306560:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784893.194286:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2356:x1874627658306560:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (588us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784893.194291:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 492 00000100:00100000:0.0:1787784893.194303:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 493 00000100:00100000:0.0:1787784893.194308:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2356:x1874627658306176:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784893.194331:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2356:x1874627658306176:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (601us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784893.194337:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 493 00000100:00080000:0.0:1787784893.194357:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784893.194361:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784893.194363:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784893.194542:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306048 00000100:00100000:2.0:1787784893.194547:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306048 00000100:00100000:2.0:1787784893.194584:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 615 00000100:00100000:2.0:1787784893.194592:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658306048:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784893.194625:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658306048:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 35us (1007us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784893.194631:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 615 00010000:00010000:2.0:1787784893.361353:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784893.483551:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967390 is committed 00000001:00080000:3.0:1787784893.483567:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967389 is committed 00010000:00010000:3.0:1787784894.386043:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784894.969063:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784894.969089:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539250780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668451584:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784894.969183:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784894.969522:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451584 00000100:00100000:2.0:1787784894.969530:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451584 00000100:00080000:0.0:1787784894.969580:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784894.969591:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1787784894.969601:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668451584:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784894.969642:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 93519872 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784894.970793:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668451584:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1191us (1607us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784894.970804:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:0.0:1787784894.971402:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539250780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668451584:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787784895.413056:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784895.694467:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784895.694684:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658306944 00000100:00100000:2.0:1787784895.694693:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658306944 00000100:00100000:2.0:1787784895.694762:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1506 00000100:00100000:2.0:1787784895.694775:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32770:x1874627658306944:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1787784895.694885:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:32770:x1874627658306944:12345-192.168.203.43@tcp:60:lhsmtool_posix.0 Request processed in 114us (437us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784895.694893:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1506 00000100:00080000:0.0:1787784895.695629:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784896.433296:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784897.009565:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539252940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668451968:0@lo:400:kworker.0 00000100:00100000:1.0:1787784897.009643:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784897.009698:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539250780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668452480:0@lo:400:kworker.0 00000100:00100000:1.0:1787784897.009727:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784897.009770:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1787784897.009776:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668451840:0@lo:400:kworker.0 00000100:00100000:1.0:1787784897.009831:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784897.009852:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668452352:0@lo:400:kworker.0 00000100:00100000:1.0:1787784897.009873:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784897.009914:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451840 00000100:00100000:1.0:1787784897.009921:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451840 00000100:00100000:1.0:1787784897.009960:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:1.0:1787784897.009969:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668451840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784897.010189:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668451840:12345-0@lo:400:kworker.0 Request processed in 220us (356us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784897.010197:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00100000:1.0:1787784897.010235:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668451840:0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010372:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668451712:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784897.010399:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784897.010414:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668452224:0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010431:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784897.010447:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784897.010452:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668452096:0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010469:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784897.010491:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451968 00000100:00100000:0.0:1787784897.010494:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451968 00000100:00100000:0.0:1787784897.010515:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668452352 00000100:00100000:0.0:1787784897.010517:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668452352 00000100:00100000:0.0:1787784897.010537:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668452096 00000100:00100000:0.0:1787784897.010539:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668452096 00000100:00100000:0.0:1787784897.010551:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:0.0:1787784897.010557:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668451968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010585:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668451968:12345-0@lo:400:kworker.0 Request processed in 29us (943us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784897.010592:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:0.0:1787784897.010606:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:0.0:1787784897.010611:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668452352:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010632:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668452352:12345-0@lo:400:kworker.0 Request processed in 21us (758us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784897.010638:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:0.0:1787784897.010646:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 496 00000100:00100000:0.0:1787784897.010651:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668452096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010676:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668452096:12345-0@lo:400:kworker.0 Request processed in 25us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784897.010682:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 496 00000100:00100000:0.0:1787784897.010698:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668452480 00000100:00100000:0.0:1787784897.010700:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668452480 00000100:00100000:0.0:1787784897.010719:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668452224 00000100:00100000:0.0:1787784897.010722:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668452224 00000100:00100000:0.0:1787784897.010747:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:0.0:1787784897.010753:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668452480:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010778:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668452480:12345-0@lo:400:kworker.0 Request processed in 26us (1050us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784897.010881:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:0.0:1787784897.010893:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:0.0:1787784897.010899:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668452224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010925:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668452224:12345-0@lo:400:kworker.0 Request processed in 27us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784897.010930:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00080000:0.0:1787784897.010955:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784897.010958:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784897.010960:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784897.010976:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668452096:0@lo:400:kworker.0 00000100:00100000:0.0:1787784897.010984:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668452224:0@lo:400:kworker.0 00000100:00100000:2.0:1787784897.011115:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668451712 00000100:00100000:2.0:1787784897.011119:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668451712 00000100:00100000:2.0:1787784897.011176:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1787784897.011183:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668451712:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784897.011208:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 91815936 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784897.011242:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668451712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (842us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784897.011248:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1787784897.011279:0:3277:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967390 gen 2 00000004:00080000:2.0:1787784897.011284:0:3277:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967390 committed transno 4294967390 00000100:00100000:2.0:1787784897.011302:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539101680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668451712:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784897.011701:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668452352:0@lo:400:kworker.0 00000100:00100000:1.0:1787784897.011711:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539252940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668451968:0@lo:400:kworker.0 00000100:00100000:1.0:1787784897.011718:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539250780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668452480:0@lo:400:kworker.0 00010000:00010000:3.0:1787784897.460997:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:3.0:1787784898.098391:0:5833:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 35 batchid updated 00000020:00080000:0.0:1787784898.098540:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 35 00000020:00080000:0.0:1787784898.098552:0:5844:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 35 00000001:02000400:3.0:1787784898.189474:0:22693:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 18:54:55 (1787784895) 00000100:00100000:0.0:1787784898.334635:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.334681:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.334719:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.334744:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784898.334747:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307072 00000100:00100000:1.0:1787784898.334758:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307072 00000100:00100000:0.0:1787784898.334767:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.334792:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784898.334828:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 617 00000100:00100000:0.0:1787784898.334839:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784898.334843:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658307072:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784898.334879:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307584 00000100:00100000:0.0:1787784898.334884:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307584 00000100:00100000:1.0:1787784898.334916:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658307072:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 78us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784898.334926:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 617 00000100:00100000:0.0:1787784898.334927:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307200 00000100:00100000:0.0:1787784898.334932:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307200 00000100:00100000:0.0:1787784898.334960:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 497 00000100:00100000:0.0:1787784898.334969:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2357:x1874627658307584:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784898.335006:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2357:x1874627658307584:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.335013:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 497 00000100:00100000:0.0:1787784898.335035:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 498 00000100:00100000:0.0:1787784898.335043:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2357:x1874627658307200:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784898.335087:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:2357:x1874627658307200:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 45us (320us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.335094:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 498 00000100:00100000:0.0:1787784898.335115:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307328 00000100:00100000:0.0:1787784898.335119:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307328 00000100:00100000:0.0:1787784898.335143:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307840 00000100:00100000:0.0:1787784898.335162:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307840 00000100:00100000:0.0:1787784898.335177:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307456 00000100:00100000:0.0:1787784898.335180:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307456 00000100:00100000:0.0:1787784898.335191:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307712 00000100:00100000:0.0:1787784898.335194:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307712 00000100:00100000:0.0:1787784898.335207:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:0.0:1787784898.335214:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658307328:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784898.335242:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658307328:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (561us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.335249:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:0.0:1787784898.335261:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 467 00000100:00100000:0.0:1787784898.335268:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658307840:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784898.335291:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658307840:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (548us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.335297:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 467 00000100:00100000:0.0:1787784898.335309:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 468 00000100:00100000:0.0:1787784898.335315:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658307456:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784898.335338:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658307456:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (546us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.335344:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 468 00000100:00100000:0.0:1787784898.335356:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00100000:0.0:1787784898.335362:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658307712:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784898.335386:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658307712:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (549us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.335393:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00080000:0.0:1787784898.335819:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784898.335823:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784898.335826:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784898.493078:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784898.588203:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784898.588355:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658307968 00000100:00100000:2.0:1787784898.588366:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658307968 00000100:00100000:2.0:1787784898.588427:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1507 00000100:00100000:2.0:1787784898.588437:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:33460:x1874627658307968:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1787784898.588544:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+50:33460:x1874627658307968:12345-192.168.203.43@tcp:59:lhsmtool_posix.0 Request processed in 109us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784898.588552:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1507 00000100:00100000:2.0:1787784898.609210:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.609310:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658308096 00000100:00100000:0.0:1787784898.609321:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658308096 00000100:00100000:0.0:1787784898.609515:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1508 00000100:00100000:0.0:1787784898.609526:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:33461:x1874627658308096:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784898.609546:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.609565:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e23 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19a6 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609627:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609644:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609655:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609670:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609676:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609682:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609964:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609972:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609978:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.609984:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.609988:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610055:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610069:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784898.610072:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.610076:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610082:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e2a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610096:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610102:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610108:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610112:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.610115:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610124:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610130:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff989520108e00) 00010000:00010000:0.0:1787784898.610132:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.610134:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610139:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e31 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610289:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e23 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19a6 expref: 108 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610299:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784898.610302:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.610306:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19a6 expref: 108 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.610316:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00100000:0.0:1787784898.610377:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:33461:x1874627658308096:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 852us (1176us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.610385:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1508 00000100:00100000:1.0:1787784898.633738:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.634116:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658308352 00000100:00100000:0.0:1787784898.634127:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658308352 00000100:00100000:0.0:1787784898.634353:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1509 00000100:00100000:0.0:1787784898.634364:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:33461:x1874627658308352:12345-192.168.203.43@tcp:36:lfs.0 00010000:00010000:0.0:1787784898.634419:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784898.634444:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc766d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1745 expref: 107 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.634464:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784898.634468:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784898.634747:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.634762:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.634772:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.634818:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.634832:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 49 pid: 10850 timeout: 789 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.634845:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668452608:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784898.634926:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787784898.641486:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784898.641725:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658308480 00000100:00100000:1.0:1787784898.641734:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658308480 00010000:00010000:1.0:1787784898.641774:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 50 pid: 10850 timeout: 789 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784898.641825:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 442 00000100:00100000:1.0:1787784898.641835:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c97c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+50:2357:x1874627658308480:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784898.641851:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784898.641858:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 50 pid: 10850 timeout: 789 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.641872:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 50 pid: 10850 timeout: 789 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.641895:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:1.0:1787784898.641900:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.641911:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.641923:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.641943:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784898.641966:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc76ac lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b175a expref: 50 pid: 10850 timeout: 789 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.641983:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668452608:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784898.641997:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c97c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+49:2357:x1874627658308480:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 164us (521us total) trans 0 rc 0/0 00010000:00010000:0.0:1787784898.642015:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.642024:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.642029:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.642051:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.642061:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.642068:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784898.642123:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 442 00000004:00200000:0.0:1787784898.643517:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00010000:00010000:0.0:1787784898.643531:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.643541:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.644292:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.644300:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.644305:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.644313:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.644329:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff98951c407300) 00010000:00010000:0.0:1787784898.644332:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.644336:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.644344:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e38 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784898.644366:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000033dd6d01 reply state 00000000e79245b1 transno 4294968343 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.644376:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784898.644761:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:33461:x1874627658308352:12345-192.168.203.43@tcp:36:lfs.0 Request processed in 10395us (11027us total) trans 4294968343 rc 0/0 00000100:00100000:0.0:1787784898.644771:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1509 00010000:00010000:1.0:1787784898.655690:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.655710:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.655718:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.655725:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.655730:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.655739:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.655754:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff989520108e00) 00010000:00010000:1.0:1787784898.655758:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.655762:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.655769:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e3f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784898.669040:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.669361:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658308608 00000100:00100000:0.0:1787784898.669371:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658308608 00000100:00100000:0.0:1787784898.669436:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1510 00000100:00100000:0.0:1787784898.669446:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+107:33462:x1874627658308608:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784898.669469:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.669491:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e46 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19ad expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.669556:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e4d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.669571:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.669588:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784898.669661:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:0.0:1787784898.669665:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 35 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784898.669698:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.669721:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e46 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ad expref: 109 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.669739:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b800/0xb0ce63137acc7e4d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ad expref: 108 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.669747:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108200) 00010000:00010000:0.0:1787784898.669750:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784898.669754:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b800, rc 0) 00000100:00100000:0.0:1787784898.669822:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:33462:x1874627658308608:12345-192.168.203.43@tcp:101:bash.0 Request processed in 378us (789us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.669829:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1510 00000100:00100000:3.0:1787784898.684985:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.685207:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658308736 00000100:00100000:0.0:1787784898.685217:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658308736 00000100:00100000:0.0:1787784898.685338:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1511 00000100:00100000:0.0:1787784898.685350:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+108:33462:x1874627658308736:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784898.685370:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.685392:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e54 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19b4 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685459:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685473:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685485:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685500:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685507:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685513:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685620:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc7e69 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685630:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc7e69 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685636:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc7e69 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784898.685667:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:0.0:1787784898.685670:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784898.685690:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc7e69 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685707:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685713:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685720:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685726:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.685730:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685738:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685749:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108200) 00010000:00010000:0.0:1787784898.685752:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.685756:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685763:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7e5b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685776:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685783:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.685789:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.689049:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.689056:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.689065:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.689083:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff989520108900) 00010000:00010000:0.0:1787784898.689086:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.689090:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.689098:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7e62 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.689173:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7e54 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19b4 expref: 110 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.689190:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134a00/0xb0ce63137acc7e69 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19b4 expref: 109 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.689197:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.689200:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787784898.689203:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539134a00, rc 0) 00000100:00100000:0.0:1787784898.689361:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539103c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:33462:x1874627658308736:12345-192.168.203.43@tcp:101:bash.0 Request processed in 4011us (4382us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.689371:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1511 00000100:00100000:1.0:1787784898.693317:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.693390:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658308864 00000100:00100000:0.0:1787784898.693400:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658308864 00000100:00100000:0.0:1787784898.693519:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1512 00000100:00100000:0.0:1787784898.693529:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:33462:x1874627658308864:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784898.693549:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.693569:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7e70 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19bb expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693625:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693637:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693647:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693697:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693704:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693710:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693771:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693777:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693783:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693789:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.693808:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693815:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693827:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.693830:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.693835:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693840:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7e77 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693853:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693859:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693864:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693869:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.693872:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693878:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693889:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff989520108800) 00010000:00010000:0.0:1787784898.693891:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.693894:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693900:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7e7e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693929:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7e70 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19bb expref: 110 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693938:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.693940:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.693943:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7e70 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19bb expref: 110 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.693951:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135e00, rc 0) 00000100:00100000:0.0:1787784898.694068:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:33462:x1874627658308864:12345-192.168.203.43@tcp:101:bash.0 Request processed in 539us (757us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.694076:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1512 00000100:00100000:3.0:1787784898.710381:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784898.710484:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658308992 00000100:00100000:2.0:1787784898.710494:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658308992 00000100:00100000:2.0:1787784898.710553:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1513 00000100:00100000:2.0:1787784898.710564:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+49:33463:x1874627658308992:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784898.710584:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784898.710604:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc7e85 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19c2 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.710662:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111c00/0xb0ce63137acc7e8c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.710675:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111c00/0xb0ce63137acc7e8c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.710686:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111c00/0xb0ce63137acc7e8c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784898.710736:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 8704 00000004:00200000:2.0:1787784898.710740:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 35 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784898.710759:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989537111c00/0xb0ce63137acc7e8c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.710778:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc7e85 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19c2 expref: 51 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.710866:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111c00/0xb0ce63137acc7e8c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19c2 expref: 50 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.710875:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108200) 00010000:00010000:2.0:1787784898.710878:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787784898.710881:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111c00, rc 0) 00000100:00100000:2.0:1787784898.710942:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+50:33463:x1874627658308992:12345-192.168.203.43@tcp:101:rm.0 Request processed in 379us (569us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784898.710949:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1513 00000100:00100000:3.0:1787784898.716332:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.716616:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658309120 00000100:00100000:0.0:1787784898.716626:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658309120 00000100:00100000:0.0:1787784898.716683:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1514 00000100:00100000:0.0:1787784898.716694:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+50:33463:x1874627658309120:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784898.716851:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784898.716874:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8000/0xb0ce63137acc7e93 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19c9 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.716935:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.716947:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.716956:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.716966:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.716970:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.716974:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717066:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7ea8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717071:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7ea8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717075:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7ea8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784898.717098:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:1.0:1787784898.717101:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784898.717122:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7ea8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717134:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717141:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717173:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717177:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.717180:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717185:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717194:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff989520108200) 00010000:00010000:1.0:1787784898.717197:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.717200:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717205:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7e9a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717214:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717218:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717221:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717225:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.717226:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717229:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717233:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff98951b73b000) 00010000:00010000:1.0:1787784898.717234:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.717236:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717239:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ea1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717259:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8000/0xb0ce63137acc7e93 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19c9 expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717267:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8800/0xb0ce63137acc7ea8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19c9 expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.717271:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784898.717272:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1787784898.717274:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8800, rc 0) 00000100:00100000:1.0:1787784898.717324:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33463:x1874627658309120:12345-192.168.203.43@tcp:101:rm.0 Request processed in 633us (1001us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784898.717331:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1514 00000100:00100000:1.0:1787784898.721542:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784898.721626:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658309248 00000100:00100000:2.0:1787784898.721692:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658309248 00000100:00100000:2.0:1787784898.721783:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1515 00000100:00100000:2.0:1787784898.721810:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33463:x1874627658309248:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784898.721829:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784898.721847:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc7eaf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19d0 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.721904:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.721916:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.721925:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.721938:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.721945:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.721950:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722005:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722012:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722018:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722024:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784898.722028:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722034:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722046:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784898.722048:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784898.722052:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722057:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc7eb6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722067:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722073:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722079:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722084:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784898.722087:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722190:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722197:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff989520131500) 00010000:00010000:2.0:1787784898.722200:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784898.722203:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722208:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc7ebd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722236:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc7eaf lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19d0 expref: 52 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722246:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784898.722248:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784898.722251:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc7eaf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19d0 expref: 52 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.722259:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111000, rc 0) 00000100:00100000:2.0:1787784898.722304:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33463:x1874627658309248:12345-192.168.203.43@tcp:101:rm.0 Request processed in 512us (770us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784898.722310:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1515 00000100:00100000:3.0:1787784898.724815:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.725784:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658309376 00000100:00100000:0.0:1787784898.725806:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658309376 00000100:00100000:0.0:1787784898.725865:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1516 00000100:00100000:0.0:1787784898.725875:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539100000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33463:x1874627658309376:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784898.725894:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.725914:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7ec4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19d7 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.725976:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.725989:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726001:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726014:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726021:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726027:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726086:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726281:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726288:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726298:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.726303:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726310:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726321:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.726324:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.726329:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726335:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7ecb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726348:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726353:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726359:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726363:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.726366:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726474:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726490:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff989520108800) 00010000:00010000:0.0:1787784898.726493:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.726496:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726503:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ed2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726539:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7ec4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19d7 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726550:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.726553:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.726557:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7ec4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19d7 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.726566:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135e00, rc 0) 00000100:00100000:0.0:1787784898.726624:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539100000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33463:x1874627658309376:12345-192.168.203.43@tcp:101:rm.0 Request processed in 750us (1816us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.726632:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1516 00000100:00100000:3.0:1787784898.737945:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784898.738096:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658309504 00000100:00100000:1.0:1787784898.738104:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658309504 00000100:00100000:1.0:1787784898.738223:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1517 00000100:00100000:1.0:1787784898.738232:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33464:x1874627658309504:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784898.738250:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784898.738266:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8000/0xb0ce63137acc7ed9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19de expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738321:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738331:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738338:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738348:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738352:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738356:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738413:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738421:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738431:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738436:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd9000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.738439:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738444:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738453:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784898.738455:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.738458:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738462:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd9000/0xb0ce63137acc7ee0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738470:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738473:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738477:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738480:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989536dd8400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.738481:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738487:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738491:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff98951b73b000) 00010000:00010000:1.0:1787784898.738493:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.738494:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738497:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8400/0xb0ce63137acc7ee7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738530:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8000/0xb0ce63137acc7ed9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19de expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738536:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784898.738537:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.738539:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989536dd8000/0xb0ce63137acc7ed9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19de expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.738545:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989536dd8000, rc 0) 00000100:00100000:1.0:1787784898.738601:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33464:x1874627658309504:12345-192.168.203.43@tcp:101:cp.0 Request processed in 371us (665us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784898.738606:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1517 00000100:00100000:3.0:1787784898.742406:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.743089:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658309632 00000100:00100000:0.0:1787784898.743141:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658309632 00000100:00100000:0.0:1787784898.743276:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1518 00000100:00100000:0.0:1787784898.743286:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33464:x1874627658309632:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784898.743304:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.743323:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7eee lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19e5 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743397:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743409:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743419:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743431:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743437:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743443:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743502:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743509:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743515:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743520:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.743525:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743536:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743548:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.743551:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.743555:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743561:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7ef5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743574:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743580:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743586:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743591:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.743594:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743600:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743607:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff989520108800) 00010000:00010000:0.0:1787784898.743609:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.743612:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743619:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7efc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743653:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7eee lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19e5 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743664:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.743666:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.743677:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135e00/0xb0ce63137acc7eee lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19e5 expref: 52 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.743690:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135e00, rc 0) 00000100:00100000:0.0:1787784898.743762:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539101680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33464:x1874627658309632:12345-192.168.203.43@tcp:101:cp.0 Request processed in 477us (1364us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.743770:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1518 00000100:00100000:2.0:1787784898.747924:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784898.747988:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658309888 00000100:00100000:1.0:1787784898.747997:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658309888 00000100:00100000:1.0:1787784898.748060:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1519 00000100:00100000:1.0:1787784898.748070:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+51:33464:x1874627658309888:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784898.748090:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784898.748111:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7f03 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19ec expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.748333:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.748348:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.748359:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.748372:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.748378:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.748383:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787784898.749901:0:10850:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1d:0x0], index 0: 0 00010000:00010000:1.0:1787784898.750215:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750274:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750288:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750327:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750334:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750341:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750347:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403641400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.750351:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750359:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750372:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784898.750375:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.750379:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750385:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641400/0xb0ce63137acc7f0a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784898.750398:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007f6a5497 reply state 00000000e293356d transno 4294968344 ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784898.750406:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787784898.750448:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403641800/0xb0ce63137acc7f03 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 54 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750467:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 53 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.750475:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:1.0:1787784898.750476:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784898.750478:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de200, rc 0) 00000100:00100000:1.0:1787784898.750538:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33464:x1874627658309888:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2469us (2621us total) trans 4294968344 rc 0/0 00000100:00100000:1.0:1787784898.750547:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1519 00010000:00010000:1.0:1787784898.751885:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.751897:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.751903:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.751909:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989403640400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.751913:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.751919:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.751930:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff98951b73bb00) 00010000:00010000:1.0:1787784898.751933:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.751937:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.751944:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403640400/0xb0ce63137acc7f11 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784898.754906:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784898.755107:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658310016 00000100:00100000:2.0:1787784898.755116:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658310016 00000100:00100000:2.0:1787784898.755350:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:2.0:1787784898.755362:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+8:2357:x1874627658310016:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784898.755388:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784898.755408:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19f3 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784898.755440:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784898.755458:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1787784898.755463:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784898.755472:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 9 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.755490:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 9 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.755501:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111e00, rc 0) 00000100:00100000:2.0:1787784898.755565:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658310016:12345-192.168.203.43@tcp:101:cp.0 Request processed in 205us (666us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784898.755572:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 441 00000100:00100000:3.0:1787784898.760272:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.760506:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658310144 00000100:00100000:0.0:1787784898.760516:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658310144 00000100:00100000:0.0:1787784898.760574:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 442 00000100:00100000:0.0:1787784898.760584:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:33464:x1874627658310144:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784898.760601:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.760619:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b19fa expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.760651:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.760667:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784898.760672:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784898.760678:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.760714:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 10 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.760729:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 10 pid: 6671 timeout: 789 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.760744:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668452736:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784898.760858:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784898.765466:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784898.765550:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658310272 00000100:00100000:2.0:1787784898.765559:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658310272 00010000:00010000:2.0:1787784898.765600:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 11 pid: 6671 timeout: 789 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784898.765650:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 443 00000100:00100000:2.0:1787784898.765659:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658310272:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784898.765682:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784898.765704:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 11 pid: 6671 timeout: 789 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.765718:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 11 pid: 6671 timeout: 789 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.765744:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784898.765748:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784898.765757:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.765768:0:21579:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.765819:0:21579:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 11 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784898.765833:0:21579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:21579:1874627668452864:192.168.203.43@tcp:105: 00000100:00100000:2.0:1787784898.765910:0:21579:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f540 going to sleep for 11 seconds 00010000:00010000:0.0:1787784898.766386:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc7f26 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19f3 expref: 11 pid: 6671 timeout: 789 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.766403:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668452736:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784898.766440:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 10 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.766451:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135e00, rc 0) 00000100:00100000:0.0:1787784898.766484:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:33464:x1874627658310144:12345-192.168.203.43@tcp:101:cp.0 Request processed in 5902us (6222us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.766496:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 442 00000100:00100000:2.0:1787784898.772653:0:21579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc03c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:21579:1874627668452864:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784898.772677:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784898.772731:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658310272:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 7073us (7273us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784898.772741:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 443 00000100:00100000:1.0:1787784898.774524:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.774586:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658310528 00000100:00100000:0.0:1787784898.774592:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658310528 00000100:00100000:0.0:1787784898.774640:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 328 00000100:00100000:0.0:1787784898.774650:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+53:33464:x1874627658310528:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:0.0:1787784898.774980:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33464:x1874627658310528:12345-192.168.203.43@tcp:35:cp.0 Request processed in 330us (461us total) trans 4294968345 rc 0/0 00000100:00100000:0.0:1787784898.774988:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 328 00000100:00100000:1.0:1787784898.811614:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.811873:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658310656 00000100:00100000:0.0:1787784898.811883:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658310656 00000100:00100000:0.0:1787784898.811942:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1520 00000100:00100000:0.0:1787784898.811953:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+109:33467:x1874627658310656:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784898.811973:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.811995:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f34 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a01 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812064:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812081:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812093:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812115:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812127:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812135:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812519:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7f49 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812529:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7f49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812537:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7f49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784898.812570:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2e:0x0]: returning size 158 00000004:00200000:0.0:1787784898.812574:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784898.812593:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7f49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812605:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7f49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812614:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812621:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812628:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812638:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.812642:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812649:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812660:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.812663:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.812668:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812675:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f3b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812688:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812695:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812701:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812707:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.812710:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812717:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812728:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff989520108800) 00010000:00010000:0.0:1787784898.812731:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.812734:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812740:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f42 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812771:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f34 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a01 expref: 111 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812785:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7f49 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a01 expref: 110 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.812923:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:0.0:1787784898.812927:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784898.812930:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150b600, rc 0) 00000100:00100000:0.0:1787784898.813036:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:33467:x1874627658310656:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1084us (1430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.813045:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1520 00000100:00100000:1.0:1787784898.819280:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.820598:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658310784 00000100:00100000:0.0:1787784898.820610:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658310784 00000100:00100000:0.0:1787784898.820671:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 443 00000100:00100000:0.0:1787784898.820682:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658310784:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784898.820700:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.820717:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7f50 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a08 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.820904:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668452992:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784898.821033:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f180 going to sleep for 11 seconds 00002000:00010000:0.0:1787784898.824773:0:13825:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1787784898.824782:0:13825:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb mtime: 0 -> 1787784898 00002000:00010000:0.0:1787784898.824849:0:13825:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb atime: 0 -> 1787784898 00002000:00010000:0.0:1787784898.824853:0:13825:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb ctime: 0 -> 1787784898 00000100:00100000:0.0:1787784898.824879:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668452992:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784898.824930:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7f50 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a08 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.824958:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc7f50 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a08 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.824973:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a200, rc 0) 00000100:00100000:0.0:1787784898.825310:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658310784:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4628us (6038us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.825319:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 443 00000100:00100000:1.0:1787784898.829745:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.829876:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658310912 00000100:00100000:0.0:1787784898.829886:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658310912 00000100:00100000:0.0:1787784898.829945:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1521 00000100:00100000:0.0:1787784898.829957:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:33467:x1874627658310912:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784898.829978:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.830000:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f57 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a0f expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.830248:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.830263:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.830275:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784898.830497:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:0.0:1787784898.830502:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 1 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784898.830531:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.830560:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f57 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 112 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.830579:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 111 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.830588:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.830591:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784898.830595:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135000, rc 0) 00000100:00100000:0.0:1787784898.830702:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33467:x1874627658310912:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 747us (964us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.830710:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1521 00000100:00100000:1.0:1787784898.834120:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.834368:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658311040 00000100:00100000:0.0:1787784898.834386:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658311040 00000100:00100000:0.0:1787784898.834443:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 444 00000100:00100000:0.0:1787784898.834453:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658311040:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784898.834472:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.834490:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a7c0d90/0xb0ce63137acc7f65 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a16 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.834547:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668453120:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784898.834877:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787784898.838537:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668453120:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784898.838585:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004a7c0d90/0xb0ce63137acc7f65 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a16 expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.838610:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a7c0d90/0xb0ce63137acc7f65 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a16 expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.838623:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135600, rc 0) 00000100:00100000:0.0:1787784898.838687:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658311040:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 4235us (4576us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.838696:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 444 00000100:00100000:1.0:1787784898.846756:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.846861:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658311296 00000100:00100000:0.0:1787784898.846871:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658311296 00000100:00100000:0.0:1787784898.846930:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1522 00000100:00100000:0.0:1787784898.846940:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33467:x1874627658311296:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:0.0:1787784898.847270:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].45, 136 off14312 00000100:00100000:0.0:1787784898.847369:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b12d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33467:x1874627658311296:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 431us (623us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.847390:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1522 00000100:00100000:1.0:1787784898.859206:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.859278:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658311424 00000100:00100000:0.0:1787784898.859289:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658311424 00000100:00100000:0.0:1787784898.859351:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1523 00000100:00100000:0.0:1787784898.859361:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33468:x1874627658311424:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784898.859381:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.859402:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f6c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a1d expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859581:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859595:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859606:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859645:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859654:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859661:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859720:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859733:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859740:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859746:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.859750:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859758:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859770:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.859773:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.859777:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859783:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f73 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859806:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859812:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859817:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859822:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.859826:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859833:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859840:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff989520108800) 00010000:00010000:0.0:1787784898.859842:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.859845:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859851:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f7a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859887:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f6c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a1d expref: 112 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859897:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.859900:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.859903:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f6c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a1d expref: 112 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.859913:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135600, rc 0) 00000100:00100000:0.0:1787784898.859974:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33468:x1874627658311424:12345-192.168.203.43@tcp:101:bash.0 Request processed in 614us (777us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.859981:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1523 00000100:00100000:1.0:1787784898.882586:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.882666:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658311552 00000100:00100000:0.0:1787784898.882675:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658311552 00000100:00100000:0.0:1787784898.882732:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1524 00000100:00100000:0.0:1787784898.882742:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33469:x1874627658311552:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784898.882767:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.882803:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f81 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a24 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882872:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882886:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882898:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882913:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882920:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882928:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882991:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.882999:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883005:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883011:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.883017:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883025:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883037:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.883040:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.883045:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883051:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7f88 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883064:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883071:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883076:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883081:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539134e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.883084:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883090:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883097:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff989520108800) 00010000:00010000:0.0:1787784898.883100:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.883103:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883109:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7f8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883188:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f81 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a24 expref: 55 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883200:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.883203:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.883207:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7f81 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a24 expref: 55 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.883219:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a200, rc 0) 00000100:00100000:0.0:1787784898.883442:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33469:x1874627658311552:12345-192.168.203.43@tcp:101:rm.0 Request processed in 700us (864us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.883450:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1524 00000100:00100000:1.0:1787784898.887987:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784898.888059:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658311680 00000100:00100000:3.0:1787784898.888069:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658311680 00000100:00100000:3.0:1787784898.888121:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1525 00000100:00100000:3.0:1787784898.888130:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33469:x1874627658311680:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784898.888180:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784898.888199:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7f96 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a2b expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888259:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888273:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888285:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888299:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888305:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888312:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888370:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888378:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888384:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888389:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784898.888393:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888400:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888412:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784898.888415:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784898.888420:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888425:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc7f9d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888438:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888444:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888449:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888454:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784898.888456:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888462:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888469:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff9895198b5900) 00010000:00010000:3.0:1787784898.888472:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784898.888475:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888480:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fa4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888511:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7f96 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a2b expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888522:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784898.888524:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784898.888528:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7f96 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a2b expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.888536:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784898.888590:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b11e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33469:x1874627658311680:12345-192.168.203.43@tcp:101:rm.0 Request processed in 462us (611us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784898.888597:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1525 00000100:00100000:2.0:1787784898.916434:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.916541:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658311808 00000100:00100000:0.0:1787784898.916552:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658311808 00000100:00100000:0.0:1787784898.916611:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1526 00000100:00100000:0.0:1787784898.916620:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33470:x1874627658311808:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784898.916640:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.916661:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7fab lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a32 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916729:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916748:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916761:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916775:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916782:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916800:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916858:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916865:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916872:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916878:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.916882:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916889:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916902:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.916906:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.916910:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916916:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc7fb2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916929:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916936:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916941:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916946:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.916949:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916955:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916962:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff989520108800) 00010000:00010000:0.0:1787784898.916964:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.916967:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.916972:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc7fb9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.917009:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7fab lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a32 expref: 55 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.917020:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.917022:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.917025:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc7fab lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a32 expref: 55 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.917034:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539134e00, rc 0) 00000100:00100000:0.0:1787784898.917095:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33470:x1874627658311808:12345-192.168.203.43@tcp:101:cp.0 Request processed in 476us (670us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.917103:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1526 00000100:00100000:2.0:1787784898.923649:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784898.923743:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658311936 00000100:00100000:1.0:1787784898.923752:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658311936 00000100:00100000:1.0:1787784898.923826:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1527 00000100:00100000:1.0:1787784898.923837:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33470:x1874627658311936:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784898.923857:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784898.923876:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc7fc0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a39 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.923946:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.923960:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.923972:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.923988:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.923995:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924001:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924063:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924069:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924075:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924080:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.924085:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924091:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924102:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784898.924105:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.924109:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924114:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc7fc7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924127:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924133:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924137:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924142:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.924186:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924193:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924200:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff98951b73bb00) 00010000:00010000:1.0:1787784898.924202:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.924205:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924210:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc7fce lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924252:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc7fc0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a39 expref: 55 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924261:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784898.924264:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.924268:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc7fc0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a39 expref: 55 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.924276:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df000, rc 0) 00000100:00100000:1.0:1787784898.924334:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33470:x1874627658311936:12345-192.168.203.43@tcp:101:cp.0 Request processed in 500us (694us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784898.924341:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1527 00000100:00100000:1.0:1787784898.930128:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784898.930516:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658312192 00000100:00100000:3.0:1787784898.930536:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658312192 00000100:00100000:3.0:1787784898.930631:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1528 00000100:00100000:3.0:1787784898.930643:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:33470:x1874627658312192:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784898.930661:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784898.930695:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7fd5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a40 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.930833:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.930849:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 110 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.930858:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.930892:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 110 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.930907:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 110 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784898.930920:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668453248:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784898.931003:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784898.936853:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.937072:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658312320 00000100:00100000:0.0:1787784898.937082:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658312320 00010000:00010000:0.0:1787784898.937124:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 111 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.937191:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 444 00000100:00100000:0.0:1787784898.937202:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658312320:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784898.937218:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784898.937225:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 111 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.937238:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 111 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.937261:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.937267:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.937277:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.937285:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.937315:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784898.937367:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658312320:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 166us (522us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.937386:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 444 00010000:00010000:3.0:1787784898.938853:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135000/0xb0ce63137acc7f5e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a0f expref: 110 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784898.938875:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668453248:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784898.938897:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939013:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939021:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939048:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939060:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939069:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784898.939595:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x20:0x0], index 1: 0 00010000:00010000:2.0:1787784898.939769:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939780:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939963:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.939999:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940006:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940013:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940019:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784898.940023:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940031:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940043:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784898.940046:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784898.940051:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940057:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc7fdc lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784898.940070:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f2b2a941 reply state 000000002bec3a5b transno 4294968346 ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784898.940078:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784898.940110:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc7fd5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 57 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940124:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 56 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784898.940131:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:2.0:1787784898.940133:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784898.940135:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110c00, rc 0) 00000100:00100000:2.0:1787784898.940228:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33470:x1874627658312192:12345-192.168.203.43@tcp:101:cp.0 Request processed in 9588us (10109us total) trans 4294968346 rc 0/0 00000100:00100000:2.0:1787784898.940236:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1528 00010000:00010000:1.0:1787784898.943862:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.943877:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.943885:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.943891:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784898.943895:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.943903:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.943919:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff989520131800) 00010000:00010000:1.0:1787784898.943923:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784898.943926:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784898.943932:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110400/0xb0ce63137acc7fe3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784898.946417:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.946531:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658312448 00000100:00100000:0.0:1787784898.946540:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658312448 00000100:00100000:0.0:1787784898.946585:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:0.0:1787784898.946593:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658312448:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784898.946607:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.946622:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a47 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784898.946642:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x20:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784898.946657:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1787784898.946660:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784898.946667:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.946681:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.946693:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98950150a200, rc 0) 00000100:00100000:0.0:1787784898.946784:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658312448:12345-192.168.203.43@tcp:101:cp.0 Request processed in 189us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.946803:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:2.0:1787784898.950043:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.950193:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658312576 00000100:00100000:0.0:1787784898.950204:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658312576 00000100:00100000:0.0:1787784898.950266:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 446 00000100:00100000:0.0:1787784898.950277:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:33470:x1874627658312576:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784898.950296:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.950539:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 2/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a4e expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.950577:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.950588:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784898.950593:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784898.950599:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 4/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.950634:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.950649:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 12 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.950665:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668453376:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784898.950809:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787784898.952819:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668453376:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784898.952864:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 3/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.952888:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989539135000, rc 0) 00000100:00100000:0.0:1787784898.952936:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:33470:x1874627658312576:12345-192.168.203.43@tcp:101:cp.0 Request processed in 2661us (2993us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.952977:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 446 00000100:00100000:2.0:1787784898.953110:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784898.953520:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658312704 00000100:00100000:3.0:1787784898.953529:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658312704 00010000:00010000:3.0:1787784898.953564:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 12 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784898.953599:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:3.0:1787784898.953611:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658312704:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784898.953626:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784898.953651:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 12 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.953665:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 12 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.953691:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b72ac33/0xb0ce63137acc7ff8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a47 expref: 12 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.953708:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784898.953713:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784898.953720:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.953730:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784898.953779:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 11 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784898.953803:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668453504:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784898.953875:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784898.956907:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.956967:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658312960 00000100:00100000:0.0:1787784898.956976:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658312960 00000100:00100000:3.0:1787784898.957054:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a903c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668453504:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784898.957092:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784898.957133:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:0.0:1787784898.957141:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:33470:x1874627658312960:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:3.0:1787784898.957177:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658312704:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 3540us (4038us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784898.957191:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 445 00000100:00100000:0.0:1787784898.958141:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33470:x1874627658312960:12345-192.168.203.43@tcp:35:cp.0 Request processed in 997us (1240us total) trans 4294968347 rc 0/0 00000100:00100000:0.0:1787784898.958172:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 329 00000100:00100000:2.0:1787784898.985951:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.988161:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658313088 00000100:00100000:0.0:1787784898.988174:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658313088 00000100:00100000:0.0:1787784898.988248:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1529 00000100:00100000:0.0:1787784898.988259:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+110:33473:x1874627658313088:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784898.988279:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.988305:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc8006 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a55 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988368:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988382:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988393:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988429:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988438:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988445:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988538:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc801b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988547:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc801b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988554:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc801b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784898.988584:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2f:0x0]: returning size 158 00000004:00200000:0.0:1787784898.988588:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784898.988605:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc801b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988621:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc801b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988630:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988636:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988642:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988648:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989539135600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.988651:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988658:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988670:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784898.988674:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.988678:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988685:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135600/0xb0ce63137acc800d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988698:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988703:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988709:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988714:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ec00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784898.988718:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988724:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988731:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff989520108b00) 00010000:00010000:0.0:1787784898.988733:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784898.988736:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988742:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8014 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988776:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539134e00/0xb0ce63137acc8006 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a55 expref: 112 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988943:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc801b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a55 expref: 111 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.988953:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:0.0:1787784898.988955:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784898.988958:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e600, rc 0) 00000100:00100000:0.0:1787784898.989059:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33473:x1874627658313088:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 801us (3116us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784898.989071:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1529 00000100:00100000:1.0:1787784898.993009:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784898.993084:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658313216 00000100:00100000:0.0:1787784898.993102:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658313216 00000100:00100000:0.0:1787784898.993360:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00100000:0.0:1787784898.993371:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2355:x1874627658313216:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784898.993389:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784898.993406:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000027190a8a/0xb0ce63137acc8022 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a5c expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784898.993459:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668453632:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784898.993537:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fd80 going to sleep for 11 seconds 00002000:00010000:0.0:1787784898.996300:0:6669:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1787784898.996309:0:6669:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb mtime: 0 -> 1787784898 00002000:00010000:0.0:1787784898.996312:0:6669:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb atime: 0 -> 1787784898 00002000:00010000:0.0:1787784898.996314:0:6669:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb ctime: 0 -> 1787784898 00000100:00100000:0.0:1787784898.996337:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668453632:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784898.996393:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000027190a8a/0xb0ce63137acc8022 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a5c expref: 37 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.996416:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000027190a8a/0xb0ce63137acc8022 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a5c expref: 37 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784898.996429:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704ec00, rc 0) 00000100:00100000:0.0:1787784898.996489:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2355:x1874627658313216:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3120us (3488us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784898.996501:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00100000:1.0:1787784899.003997:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.004785:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658313344 00000100:00100000:3.0:1787784899.004807:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658313344 00000100:00100000:3.0:1787784899.004932:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1530 00000100:00100000:3.0:1787784899.004943:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33473:x1874627658313344:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784899.004963:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.004985:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8029 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a63 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.005046:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.005125:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.005139:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784899.005221:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:3.0:1787784899.005225:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784899.005256:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.005283:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8029 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 113 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.005302:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 112 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.005311:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.005314:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784899.005318:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6800, rc 0) 00000100:00100000:3.0:1787784899.005392:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33473:x1874627658313344:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 451us (1405us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.005404:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1530 00000100:00100000:1.0:1787784899.008880:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.008987:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658313472 00000100:00100000:0.0:1787784899.008996:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658313472 00000100:00100000:0.0:1787784899.009234:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 448 00000100:00100000:0.0:1787784899.009248:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658313472:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784899.009265:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.009284:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000027190a8a/0xb0ce63137acc8037 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a6a expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.009347:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668453760:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784899.009418:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784899.015741:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668453760:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784899.015863:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000027190a8a/0xb0ce63137acc8037 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a6a expref: 37 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.015890:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000027190a8a/0xb0ce63137acc8037 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a6a expref: 37 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.015905:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704ec00, rc 0) 00000100:00100000:0.0:1787784899.015965:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658313472:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 6720us (7093us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.015974:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 448 00000100:00100000:1.0:1787784899.031409:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.031610:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658313728 00000100:00100000:2.0:1787784899.031675:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658313728 00000100:00100000:2.0:1787784899.031766:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1531 00000100:00100000:2.0:1787784899.031777:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33473:x1874627658313728:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:2.0:1787784899.032022:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].46, 136 off14448 00000100:00100000:2.0:1787784899.032378:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33473:x1874627658313728:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 330us (707us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784899.032390:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1531 00000100:00100000:1.0:1787784899.039524:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.039612:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658313856 00000100:00100000:0.0:1787784899.039621:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658313856 00000100:00100000:0.0:1787784899.039953:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1532 00000100:00100000:0.0:1787784899.039963:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33474:x1874627658313856:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784899.040821:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.040843:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc803e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a71 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.040913:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.040928:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.040939:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.040953:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.040960:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.040966:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041085:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041097:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041104:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041115:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.041119:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041127:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041138:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.041141:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.041145:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041179:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc8045 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041192:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041199:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041205:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041210:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.041213:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041218:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041230:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff989520108b00) 00010000:00010000:0.0:1787784899.041233:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.041236:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041242:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc804c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041274:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc803e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a71 expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041284:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.041286:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.041289:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc803e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a71 expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.041299:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704ec00, rc 0) 00000100:00100000:1.0:1787784899.041582:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33474:x1874627658313856:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1619us (2065us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.041591:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1532 00000100:00100000:3.0:1787784899.050330:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.051063:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658313984 00000100:00100000:0.0:1787784899.051073:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658313984 00000100:00100000:0.0:1787784899.051201:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1533 00000100:00100000:0.0:1787784899.051212:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33475:x1874627658313984:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784899.051284:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.051307:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8053 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a78 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051380:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051394:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051405:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051419:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051431:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051437:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051494:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051502:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051508:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051524:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.051528:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051537:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051548:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.051551:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.051555:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051561:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc805a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051573:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051581:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051587:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051592:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.051595:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051602:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051608:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff989520131b00) 00010000:00010000:2.0:1787784899.051611:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.051613:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051618:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8061 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051650:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8053 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a78 expref: 58 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051660:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.051663:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.051666:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8053 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a78 expref: 58 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.051681:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111a00, rc 0) 00000100:00100000:2.0:1787784899.051739:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33475:x1874627658313984:12345-192.168.203.43@tcp:101:rm.0 Request processed in 528us (1417us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784899.051747:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1533 00000100:00100000:2.0:1787784899.054180:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.054258:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658314112 00000100:00100000:1.0:1787784899.054267:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658314112 00000100:00100000:1.0:1787784899.054320:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1534 00000100:00100000:1.0:1787784899.054329:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33475:x1874627658314112:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784899.054347:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.054364:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8068 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a7f expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054429:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054443:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054455:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054468:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054476:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054483:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054535:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054541:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054547:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054553:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.054557:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054564:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054574:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.054577:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.054581:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054587:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc806f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054597:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054604:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054610:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054615:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.054617:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054623:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054629:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff98951b73bb00) 00010000:00010000:1.0:1787784899.054631:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.054634:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054640:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8076 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054904:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8068 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a7f expref: 58 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054915:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.054918:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.054921:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8068 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a7f expref: 58 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.054936:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df800, rc 0) 00000100:00100000:1.0:1787784899.055026:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33475:x1874627658314112:12345-192.168.203.43@tcp:101:rm.0 Request processed in 697us (852us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.055034:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1534 00000100:00100000:1.0:1787784899.070643:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.070723:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658314240 00000100:00100000:2.0:1787784899.070733:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658314240 00000100:00100000:2.0:1787784899.070832:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1535 00000100:00100000:2.0:1787784899.070842:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33476:x1874627658314240:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784899.070861:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.070880:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc807d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a86 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.070944:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.070958:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.070968:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.070994:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071001:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071011:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071081:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071088:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071094:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071101:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.071105:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071113:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071124:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.071127:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.071132:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071138:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8084 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071188:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071195:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071201:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071206:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.071216:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071223:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071231:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff989520131300) 00010000:00010000:2.0:1787784899.071233:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.071236:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071242:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc808b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071273:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc807d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a86 expref: 58 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071283:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.071285:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.071289:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc807d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a86 expref: 58 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.071298:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111a00, rc 0) 00000100:00100000:2.0:1787784899.071362:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33476:x1874627658314240:12345-192.168.203.43@tcp:101:cp.0 Request processed in 520us (727us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784899.071380:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1535 00000100:00100000:2.0:1787784899.074281:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.074355:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658314368 00000100:00100000:3.0:1787784899.074364:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658314368 00000100:00100000:3.0:1787784899.074417:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1536 00000100:00100000:3.0:1787784899.074427:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33476:x1874627658314368:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784899.074446:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.074464:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8092 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a8d expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.074591:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.074606:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.074617:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.074632:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.074639:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.074656:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075329:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075339:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075346:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075352:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.075357:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075365:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075378:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.075381:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.075386:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075392:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8099 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075405:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075411:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075417:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075423:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.075426:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075433:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075441:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff9895198b5900) 00010000:00010000:3.0:1787784899.075443:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.075446:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075452:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc80a0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075487:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8092 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a8d expref: 58 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075498:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.075500:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.075504:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8092 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a8d expref: 58 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.075514:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784899.075915:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33476:x1874627658314368:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1488us (1641us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784899.075925:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1536 00000100:00100000:3.0:1787784899.080939:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.081405:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658314624 00000100:00100000:1.0:1787784899.081414:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658314624 00000100:00100000:1.0:1787784899.081510:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1537 00000100:00100000:1.0:1787784899.081580:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:33476:x1874627658314624:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784899.081601:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.081621:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80a7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a94 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.081905:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.081920:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 111 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.081929:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.081960:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 111 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.086086:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 111 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.086117:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668453888:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784899.086311:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd19c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784899.088567:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668453888:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784899.088597:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.088614:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.088646:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658314752 00000100:00100000:2.0:1787784899.088653:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658314752 00010000:00010000:2.0:1787784899.088699:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 112 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.088801:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 446 00000100:00100000:2.0:1787784899.088813:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:2357:x1874627658314752:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784899.088827:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784899.088833:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 112 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.088845:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 112 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.088865:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8030 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a63 expref: 112 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.088879:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.088896:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.088909:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.088915:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.088937:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784899.088981:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658314752:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 171us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784899.088986:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 446 00010000:00010000:1.0:1787784899.089042:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.089047:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.089067:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.089076:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.089085:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787784899.089840:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1e:0x0], index 0: 0 00010000:00010000:1.0:1787784899.090000:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090010:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090018:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090178:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090190:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090197:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090203:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.090207:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090214:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090224:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.090227:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.090231:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090238:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80ae lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787784899.090254:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c203ae89 reply state 000000001a09bfe5 transno 4294968348 ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.090262:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787784899.090294:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80a7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 60 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090308:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.090320:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:1.0:1787784899.090323:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784899.090326:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dec00, rc 0) 00000100:00100000:1.0:1787784899.090385:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33476:x1874627658314624:12345-192.168.203.43@tcp:101:cp.0 Request processed in 8862us (9452us total) trans 4294968348 rc 0/0 00000100:00100000:1.0:1787784899.090393:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1537 00010000:00010000:1.0:1787784899.091704:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.091717:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.091727:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.091733:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dea00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.091737:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.091745:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.091756:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff98951b73bb00) 00010000:00010000:1.0:1787784899.091758:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.091762:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.091768:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80b5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.093539:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.093616:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658314880 00000100:00100000:0.0:1787784899.093624:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658314880 00000100:00100000:0.0:1787784899.093677:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 449 00000100:00100000:0.0:1787784899.093687:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658314880:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.093705:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.093727:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1a9b expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.093753:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784899.093771:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1787784899.093775:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.093797:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 10 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.093817:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 10 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.093829:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704ec00, rc 0) 00000100:00100000:0.0:1787784899.093883:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658314880:12345-192.168.203.43@tcp:101:cp.0 Request processed in 198us (353us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.093891:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 449 00000100:00100000:3.0:1787784899.099000:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.099069:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658315008 00000100:00100000:0.0:1787784899.099079:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658315008 00000100:00100000:0.0:1787784899.099203:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 450 00000100:00100000:0.0:1787784899.099214:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527deb450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:33476:x1874627658315008:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.099230:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.099245:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1aa2 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.099269:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.099280:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.099283:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.099286:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 11 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.099313:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.099327:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 11 pid: 13825 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.099341:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668454016:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784899.099430:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787784899.106422:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.106510:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658315136 00000100:00100000:3.0:1787784899.106519:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658315136 00000100:00100000:0.0:1787784899.106532:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668454016:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784899.106554:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 12 pid: 13825 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.106569:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.106585:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e400, rc 0) 00000100:00100000:3.0:1787784899.106600:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00100000:3.0:1787784899.106608:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658315136:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784899.106620:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784899.106639:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 12 pid: 13825 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.106640:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527deb450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:33476:x1874627658315008:12345-192.168.203.43@tcp:101:cp.0 Request processed in 7429us (7651us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.106648:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 450 00010000:00010000:3.0:1787784899.106652:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 12 pid: 13825 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.106674:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc80ca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a9b expref: 11 pid: 13825 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.106687:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.106691:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.106698:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 10 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.106708:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 10 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.106738:0:11303:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 10 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.106750:0:11303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668454144:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784899.106809:0:11303:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787784899.111205:0:11303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668454144:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784899.111561:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784899.111618:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2356:x1874627658315136:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5008us (5203us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.111629:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 447 00000100:00100000:2.0:1787784899.112772:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.112835:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658315392 00000100:00100000:1.0:1787784899.112843:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658315392 00000100:00100000:1.0:1787784899.112899:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:1.0:1787784899.112909:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+59:33476:x1874627658315392:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784899.113509:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33476:x1874627658315392:12345-192.168.203.43@tcp:35:cp.0 Request processed in 600us (738us total) trans 4294968349 rc 0/0 00000100:00100000:1.0:1787784899.113518:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 330 00000100:00100000:2.0:1787784899.144384:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.144479:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658315520 00000100:00100000:1.0:1787784899.144488:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658315520 00000100:00100000:1.0:1787784899.144546:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1538 00000100:00100000:1.0:1787784899.144555:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:33479:x1874627658315520:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784899.144575:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.144594:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1aa9 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.144657:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.144671:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.144683:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.144697:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.144703:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.144710:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.145962:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de600/0xb0ce63137acc80ed lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146258:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de600/0xb0ce63137acc80ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146271:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de600/0xb0ce63137acc80ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784899.146313:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x30:0x0]: returning size 158 00000004:00200000:1.0:1787784899.146317:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784899.146335:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de600/0xb0ce63137acc80ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146346:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de600/0xb0ce63137acc80ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146354:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146360:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146366:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146372:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.146376:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146384:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146395:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.146398:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.146402:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146407:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc80df lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146421:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146426:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146432:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146437:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.146440:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146446:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146452:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff98951b73bb00) 00010000:00010000:1.0:1787784899.146455:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.146457:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146463:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146502:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa9 expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146514:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de600/0xb0ce63137acc80ed lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa9 expref: 112 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.146521:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:1.0:1787784899.146523:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784899.146526:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de600, rc 0) 00000100:00100000:1.0:1787784899.146597:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f09e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33479:x1874627658315520:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2042us (2222us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.146606:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1538 00000100:00100000:2.0:1787784899.158307:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.158634:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.158720:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658315776 00000100:00100000:1.0:1787784899.158729:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658315776 00000100:00100000:1.0:1787784899.158793:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 75 00000100:00100000:1.0:1787784899.158804:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:2357:x1874627658315776:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784899.159005:0:13404:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.159270:0:13404:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ab7 expref: -99 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159331:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159345:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159357:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159371:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159378:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159385:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159446:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159452:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159458:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159463:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.159467:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159475:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159486:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.159489:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.159493:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159500:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc80fb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159753:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159761:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159766:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159771:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.159773:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.159779:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.160317:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658315648 00000100:00100000:0.0:1787784899.160325:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658315648 00000100:00100000:0.0:1787784899.160374:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:0.0:1787784899.160383:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658315648:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784899.160394:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.160406:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a7fead7/0xb0ce63137acc8109 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ab0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.160453:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668454272:192.168.203.43@tcp:106: 00000100:00100000:0.0:1787784899.160637:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fd80 going to sleep for 11 seconds 00010000:00010000:1.0:1787784899.161471:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff98951b73bb00) 00010000:00010000:1.0:1787784899.161476:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.161481:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.161488:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8102 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.161542:0:13404:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80f4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ab7 expref: 113 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.161558:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.161560:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784899.161564:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc80f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ab7 expref: 113 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.161574:0:13404:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dea00, rc -2) 00000100:00100000:1.0:1787784899.161636:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:2357:x1874627658315776:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 2833us (3001us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784899.161645:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 75 00002000:00010000:0.0:1787784899.174414:0:10688:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1787784899.174423:0:10688:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb mtime: 0 -> 1787784898 00002000:00010000:0.0:1787784899.174426:0:10688:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb atime: 0 -> 1787784898 00002000:00010000:0.0:1787784899.174428:0:10688:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb ctime: 0 -> 1787784898 00000100:00100000:0.0:1787784899.174450:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668454272:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784899.174499:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000a7fead7/0xb0ce63137acc8109 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ab0 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.174522:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a7fead7/0xb0ce63137acc8109 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ab0 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.174535:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f200, rc 0) 00000100:00100000:0.0:1787784899.174595:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658315648:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 14213us (16298us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.174603:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:2.0:1787784899.183525:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.183601:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658315904 00000100:00100000:3.0:1787784899.183611:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658315904 00000100:00100000:3.0:1787784899.183678:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1539 00000100:00100000:3.0:1787784899.183688:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33479:x1874627658315904:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784899.183708:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.183730:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8110 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1aef expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.183810:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.183824:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.183836:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784899.183893:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:3.0:1787784899.183897:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 3 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784899.183917:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.183937:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8110 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 114 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.183950:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 113 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.183957:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.183959:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784899.183961:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784899.184116:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33479:x1874627658315904:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 428us (598us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.184123:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1539 00000100:00100000:3.0:1787784899.187507:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.187586:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658316032 00000100:00100000:0.0:1787784899.187596:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658316032 00000100:00100000:0.0:1787784899.187650:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:0.0:1787784899.187661:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658316032:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784899.187970:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.187998:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a7fead7/0xb0ce63137acc811e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1af6 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.188560:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668454400:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784899.188693:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784899.200103:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668454400:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784899.200278:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000a7fead7/0xb0ce63137acc811e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1af6 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.200305:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a7fead7/0xb0ce63137acc811e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1af6 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.200319:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f200, rc 0) 00000100:00100000:1.0:1787784899.200431:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658316032:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 12771us (13061us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.200440:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:2.0:1787784899.207911:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.207992:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658316288 00000100:00100000:1.0:1787784899.208002:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658316288 00000100:00100000:1.0:1787784899.208065:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1540 00000100:00100000:1.0:1787784899.208076:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33479:x1874627658316288:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:1.0:1787784899.208952:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].47, 136 off14584 00000100:00100000:1.0:1787784899.209091:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33479:x1874627658316288:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 1015us (1187us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.209100:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1540 00000100:00100000:2.0:1787784899.216888:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.216934:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.217033:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658316416 00000100:00100000:1.0:1787784899.217042:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658316416 00000100:00100000:1.0:1787784899.217113:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1541 00000100:00100000:1.0:1787784899.217123:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33482:x1874627658316416:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784899.217180:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.217201:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc8125 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1afd expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217266:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217281:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217293:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217307:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217313:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217323:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217396:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217403:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217408:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217414:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.217418:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217424:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217435:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.217438:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.217442:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217448:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc812c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217460:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217465:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217471:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217476:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfa00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.217478:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217484:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217490:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff98951b73bb00) 00010000:00010000:1.0:1787784899.217492:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.217495:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217501:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc8133 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217533:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc8125 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1afd expref: 114 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217544:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.217546:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.217550:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc8125 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1afd expref: 114 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.217559:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dea00, rc 0) 00000100:00100000:1.0:1787784899.217614:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33482:x1874627658316416:12345-192.168.203.43@tcp:101:bash.0 Request processed in 492us (734us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.217625:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1541 00000100:00100000:3.0:1787784899.220453:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658316544 00000100:00100000:3.0:1787784899.220467:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658316544 00000100:00100000:1.0:1787784899.220601:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 76 00000100:00100000:1.0:1787784899.220612:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539378780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:2357:x1874627658316544:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784899.220646:0:5816:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.220670:0:5816:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc813a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b04 expref: -99 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220721:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220734:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220745:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220759:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220766:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220772:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220843:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220849:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220860:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220865:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.220869:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220876:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220887:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.220890:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.220895:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220901:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8141 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220914:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220920:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220925:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220930:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.220933:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220943:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220950:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x996ea506 (ffff98951b73b700) 00010000:00010000:1.0:1787784899.220952:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.220954:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220960:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc8148 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.220994:0:5816:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc813a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b04 expref: 114 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.221004:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.221006:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784899.221010:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc813a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b04 expref: 114 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.221019:0:5816:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df200, rc -2) 00000100:00100000:1.0:1787784899.221102:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539378780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:2357:x1874627658316544:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 491us (4166us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784899.221110:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 76 00000100:00100000:2.0:1787784899.232067:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.232199:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658316672 00000100:00100000:3.0:1787784899.232208:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658316672 00000100:00100000:3.0:1787784899.232312:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1542 00000100:00100000:3.0:1787784899.232323:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33483:x1874627658316672:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784899.232346:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.232367:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc814f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b0b expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232427:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232445:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232455:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232469:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232476:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232484:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232542:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232549:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232555:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232561:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.232565:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232573:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232584:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.232588:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.232592:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232598:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8156 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232611:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232617:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232622:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232627:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.232630:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232636:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232642:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff9895198b5900) 00010000:00010000:3.0:1787784899.232645:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.232647:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232653:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc815d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232686:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc814f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b0b expref: 61 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232696:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.232699:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.232703:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc814f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b0b expref: 61 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.232712:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784899.232777:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33483:x1874627658316672:12345-192.168.203.43@tcp:101:rm.0 Request processed in 456us (720us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784899.232784:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1542 00000100:00100000:3.0:1787784899.236193:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.236400:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658316800 00000100:00100000:1.0:1787784899.236410:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658316800 00000100:00100000:1.0:1787784899.236466:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1543 00000100:00100000:1.0:1787784899.236475:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33483:x1874627658316800:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784899.236499:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.237783:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8164 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b12 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241020:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241043:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241055:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241072:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241079:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241086:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241751:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241760:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241766:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241771:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.241775:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241781:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241802:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.241806:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.241810:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241815:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc816b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241828:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241834:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241840:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241844:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.241847:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241853:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241860:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff98951b73b700) 00010000:00010000:1.0:1787784899.241863:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.241867:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241873:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc8172 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241911:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8164 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b12 expref: 61 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241922:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.241924:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.241927:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8164 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b12 expref: 61 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.241936:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df200, rc 0) 00000100:00100000:1.0:1787784899.242184:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33483:x1874627658316800:12345-192.168.203.43@tcp:101:rm.0 Request processed in 5707us (5996us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.242193:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1543 00000100:00100000:2.0:1787784899.266393:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.267269:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658316928 00000100:00100000:0.0:1787784899.267279:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658316928 00000100:00100000:0.0:1787784899.267434:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1544 00000100:00100000:0.0:1787784899.267454:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539379e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33484:x1874627658316928:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.267475:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.267496:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8179 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b19 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267597:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267620:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267653:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267676:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267685:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267691:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267767:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267775:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267781:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267806:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.267810:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267818:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267829:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.267832:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.267836:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267842:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc8180 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267855:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267861:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267867:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267873:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.267876:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267882:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267889:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff989520108c00) 00010000:00010000:0.0:1787784899.267892:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.267895:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267900:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8187 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267936:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8179 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b19 expref: 61 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267947:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.267949:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.267953:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8179 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b19 expref: 61 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.267962:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e000, rc 0) 00000100:00100000:0.0:1787784899.268030:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539379e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33484:x1874627658316928:12345-192.168.203.43@tcp:101:cp.0 Request processed in 587us (1646us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.268039:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1544 00000100:00100000:2.0:1787784899.275534:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.275779:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658317056 00000100:00100000:3.0:1787784899.275830:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658317056 00000100:00100000:3.0:1787784899.275910:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1545 00000100:00100000:3.0:1787784899.275921:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33484:x1874627658317056:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784899.275948:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.275976:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc818e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b20 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276064:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276085:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276105:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276125:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276133:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276187:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276272:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276287:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276295:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276301:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.276306:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276314:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276326:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.276329:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.276333:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276340:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6c00/0xb0ce63137acc8195 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276353:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276359:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276374:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276379:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.276382:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276388:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276402:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff9895198b5900) 00010000:00010000:3.0:1787784899.276404:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.276407:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276414:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc819c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276454:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc818e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b20 expref: 61 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276474:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.276476:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.276481:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc818e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b20 expref: 61 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.276491:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784899.276571:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33484:x1874627658317056:12345-192.168.203.43@tcp:101:cp.0 Request processed in 644us (1036us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784899.276579:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1545 00000100:00100000:3.0:1787784899.285296:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.285604:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658317312 00000100:00100000:1.0:1787784899.285615:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658317312 00000100:00100000:1.0:1787784899.285740:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1546 00000100:00100000:1.0:1787784899.285751:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+60:33484:x1874627658317312:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784899.285772:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.285809:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc81a3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b27 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.285943:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.285964:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 112 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.285973:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.286007:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 112 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.286020:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 112 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.286033:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668454528:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784899.286121:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab46c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784899.291222:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668454528:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784899.291253:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.296302:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.296404:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658317440 00000100:00100000:3.0:1787784899.296414:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658317440 00010000:00010000:3.0:1787784899.296456:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 113 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.296486:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 448 00000100:00100000:3.0:1787784899.296495:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:2357:x1874627658317440:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784899.296509:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784899.296515:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 113 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.296528:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 113 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.296549:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8117 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aef expref: 113 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.296566:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.296573:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.296582:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.296590:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.296624:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784899.296684:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:2357:x1874627658317440:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 190us (390us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.296692:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 448 00010000:00010000:2.0:1787784899.296827:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.296836:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.296860:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.296869:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.296877:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784899.297521:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x21:0x0], index 1: 0 00010000:00010000:2.0:1787784899.297682:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297693:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297702:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297738:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297746:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297752:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297757:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.297762:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297770:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297914:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.297918:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.297923:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.297930:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de000/0xb0ce63137acc81aa lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784899.297943:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001751bc2c reply state 00000000e8b3d0f4 transno 4294968350 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.297952:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784899.297986:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc81a3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 63 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.298002:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 62 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.298009:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:2.0:1787784899.298011:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784899.298013:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110000, rc 0) 00000100:00100000:2.0:1787784899.298056:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33484:x1874627658317312:12345-192.168.203.43@tcp:101:cp.0 Request processed in 12307us (12771us total) trans 4294968350 rc 0/0 00000100:00100000:2.0:1787784899.298063:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1546 00010000:00010000:1.0:1787784899.300269:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.300285:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.300293:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.300300:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.300305:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.300313:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.300329:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff989520131300) 00010000:00010000:1.0:1787784899.300332:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.300337:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.300343:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc81b1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.305322:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.305381:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.305986:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658317568 00000100:00100000:0.0:1787784899.305997:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658317568 00000100:00100000:0.0:1787784899.306049:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 453 00000100:00100000:0.0:1787784899.306059:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658317568:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.306076:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.306097:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b2e expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.306125:0:6669:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x21:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784899.306165:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1787784899.306170:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.306180:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.306199:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 12 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.306213:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e000, rc 0) 00000100:00100000:0.0:1787784899.306280:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658317568:12345-192.168.203.43@tcp:101:cp.0 Request processed in 221us (969us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.306287:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 453 00000100:00100000:2.0:1787784899.307972:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658317696 00000100:00100000:2.0:1787784899.307988:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658317696 00000100:00100000:2.0:1787784899.308057:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 77 00000100:00100000:2.0:1787784899.308067:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+62:2355:x1874627658317696:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:2.0:1787784899.308130:0:13404:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.308173:0:13404:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc81cd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b35 expref: -99 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308235:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308249:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308261:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308276:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308283:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308290:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308361:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308384:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308397:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308404:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.308408:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308417:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308430:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.308433:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.308438:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308445:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc81d4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308459:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308465:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308472:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308478:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.308481:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308491:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308499:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff989520131600) 00010000:00010000:2.0:1787784899.308502:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.308504:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308510:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc81db lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308543:0:13404:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc81cd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b35 expref: 63 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308554:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.308556:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787784899.308561:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc81cd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b35 expref: 63 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.308571:0:13404:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110800, rc -2) 00000100:00100000:2.0:1787784899.308635:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+62:2355:x1874627658317696:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 570us (3263us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787784899.308644:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 77 00000100:00100000:1.0:1787784899.320816:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.321027:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658317824 00000100:00100000:0.0:1787784899.321039:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658317824 00000100:00100000:0.0:1787784899.321103:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 454 00000100:00100000:0.0:1787784899.321115:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:33484:x1874627658317824:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.321137:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.321201:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b6d expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.321236:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.321245:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.321250:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.321255:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.321289:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 13 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.321305:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 13 pid: 6669 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.321319:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403048000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668454656:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784899.321413:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f300 going to sleep for 11 seconds 00000100:00100000:0.0:1787784899.336907:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.336977:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658317952 00000100:00100000:3.0:1787784899.336985:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658317952 00010000:00010000:3.0:1787784899.337022:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 14 pid: 6669 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.337060:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 449 00000100:00100000:3.0:1787784899.337068:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c95050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658317952:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784899.337082:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784899.337103:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 14 pid: 6669 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.337115:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 14 pid: 6669 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.337143:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.337177:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.337184:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 14 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.337194:0:21579:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 14 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.337226:0:21579:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 14 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.337238:0:21579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a93840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:21579:1874627668454784:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784899.337303:0:21579:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169b40 going to sleep for 11 seconds 00010000:00010000:2.0:1787784899.337454:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002038a015/0xb0ce63137acc81c6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b2e expref: 14 pid: 6669 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.337469:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403048000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668454656:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784899.337500:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 13 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.337509:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f400, rc 0) 00000100:00100000:2.0:1787784899.337534:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:33484:x1874627658317824:12345-192.168.203.43@tcp:101:cp.0 Request processed in 16422us (16732us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784899.337541:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 454 00000100:00100000:3.0:1787784899.339660:0:21579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a93840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:21579:1874627668454784:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784899.339682:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784899.339730:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c95050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658317952:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2662us (2830us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.339741:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 449 00000100:00100000:0.0:1787784899.347075:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.347199:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658318208 00000100:00100000:1.0:1787784899.347208:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658318208 00000100:00100000:1.0:1787784899.347255:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00100000:1.0:1787784899.347263:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+62:33484:x1874627658318208:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784899.347608:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33484:x1874627658318208:12345-192.168.203.43@tcp:35:cp.0 Request processed in 344us (541us total) trans 4294968351 rc 0/0 00000100:00100000:1.0:1787784899.347617:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 331 00000100:00100000:0.0:1787784899.383893:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.383984:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658318336 00000100:00100000:1.0:1787784899.383994:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658318336 00000100:00100000:1.0:1787784899.384049:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 78 00000100:00100000:1.0:1787784899.384059:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+63:2354:x1874627658318336:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784899.384110:0:5816:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.384128:0:5816:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc81e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1b74 expref: -99 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384211:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384229:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384239:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384254:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384261:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384268:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384328:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384335:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384342:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384348:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.384351:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384359:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384370:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.384374:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.384378:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384384:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc81f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384396:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384402:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384408:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384413:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.384416:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384422:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384429:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff98951b73b700) 00010000:00010000:1.0:1787784899.384431:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.384434:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384439:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc81f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384469:0:5816:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc81e9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b74 expref: 64 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384478:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.384481:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784899.384484:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc81e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b74 expref: 64 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.384494:0:5816:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df400, rc -2) 00000100:00100000:1.0:1787784899.384549:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+63:2354:x1874627658318336:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 492us (665us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784899.384558:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 78 00000100:00100000:2.0:1787784899.404416:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.404561:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658318464 00000100:00100000:0.0:1787784899.404571:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658318464 00000100:00100000:0.0:1787784899.404633:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1547 00000100:00100000:0.0:1787784899.404644:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+112:33491:x1874627658318464:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784899.404664:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.404699:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc81fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bac expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.404764:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.404778:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.404953:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.404968:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.404976:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.404982:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405073:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8213 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405080:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8213 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405087:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8213 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784899.405116:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x31:0x0]: returning size 158 00000004:00200000:0.0:1787784899.405120:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784899.405136:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8213 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405189:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8213 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405198:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405205:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405211:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405218:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.405222:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405230:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405240:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.405244:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.405248:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405254:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8205 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405267:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405273:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405279:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405284:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.405287:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405292:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405298:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff989520108a00) 00010000:00010000:0.0:1787784899.405300:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.405303:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405308:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc820c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405339:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc81fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bac expref: 114 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405352:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8213 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bac expref: 113 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.405358:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:0.0:1787784899.405360:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784899.405363:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fa00, rc 0) 00000100:00100000:0.0:1787784899.405578:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33491:x1874627658318464:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 936us (1171us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.405590:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1547 00000100:00100000:0.0:1787784899.410124:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.410270:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658318592 00000100:00100000:1.0:1787784899.410280:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658318592 00000100:00100000:1.0:1787784899.410336:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 455 00000100:00100000:1.0:1787784899.410347:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2355:x1874627658318592:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784899.410363:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.410390:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000321934e4/0xb0ce63137acc821a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bb3 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.410446:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539250000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668454912:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784899.410533:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4480 going to sleep for 11 seconds 00002000:00010000:1.0:1787784899.413262:0:13825:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb size: 0 -> 158 00002000:00010000:1.0:1787784899.413273:0:13825:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb mtime: 0 -> 1787784898 00002000:00010000:1.0:1787784899.413275:0:13825:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb atime: 0 -> 1787784898 00002000:00010000:1.0:1787784899.413278:0:13825:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb ctime: 0 -> 1787784898 00000100:00100000:1.0:1787784899.413300:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539250000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668454912:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784899.413336:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000321934e4/0xb0ce63137acc821a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bb3 expref: 37 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.413360:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000321934e4/0xb0ce63137acc821a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bb3 expref: 37 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.413373:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df400, rc 0) 00000100:00100000:1.0:1787784899.413439:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2355:x1874627658318592:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3094us (3325us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.413447:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 455 00000100:00100000:2.0:1787784899.420633:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.420765:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658318720 00000100:00100000:1.0:1787784899.420775:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658318720 00000100:00100000:1.0:1787784899.420844:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1548 00000100:00100000:1.0:1787784899.420855:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33491:x1874627658318720:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784899.420876:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.420897:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8221 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bba expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.420964:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.420979:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.420990:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784899.421046:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:1.0:1787784899.421049:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 4 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784899.421072:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.421098:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8221 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 115 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.421115:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 114 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.421123:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.421125:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784899.421129:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfe00, rc 0) 00000100:00100000:1.0:1787784899.421228:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33491:x1874627658318720:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 374us (603us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.421235:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1548 00000100:00100000:0.0:1787784899.427138:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.427277:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658318848 00000100:00100000:3.0:1787784899.427285:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658318848 00000100:00100000:3.0:1787784899.427343:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 456 00000100:00100000:3.0:1787784899.427352:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658318848:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784899.427378:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.427397:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc822f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bc1 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.427453:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668455040:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784899.427540:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169480 going to sleep for 11 seconds 00000100:00100000:3.0:1787784899.430241:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668455040:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784899.430279:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc822f lrc: 1/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bc1 expref: 37 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.430302:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc822f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bc1 expref: 37 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.430316:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784899.430364:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658318848:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3013us (3235us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784899.430386:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 456 00000100:00100000:2.0:1787784899.434950:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.435039:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658319104 00000100:00100000:1.0:1787784899.435049:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658319104 00000100:00100000:1.0:1787784899.435108:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1549 00000100:00100000:1.0:1787784899.435118:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33491:x1874627658319104:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:1.0:1787784899.435400:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].48, 136 off14720 00000100:00100000:1.0:1787784899.435481:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33491:x1874627658319104:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 364us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.435488:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1549 00000100:00100000:0.0:1787784899.455452:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.455553:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658319232 00000100:00100000:1.0:1787784899.455562:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658319232 00000100:00100000:1.0:1787784899.455622:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1550 00000100:00100000:1.0:1787784899.455633:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33492:x1874627658319232:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784899.455653:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.455684:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8236 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bc8 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455752:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455765:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455775:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455802:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455810:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455816:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455873:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455880:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455885:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455890:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.455894:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455905:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455914:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.455918:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.455922:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455928:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc823d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455940:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455946:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455952:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455957:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.455960:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455966:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455973:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff98951b73b700) 00010000:00010000:1.0:1787784899.455976:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.455979:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.455985:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8244 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.456017:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8236 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bc8 expref: 115 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.456027:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.456029:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.456032:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8236 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bc8 expref: 115 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.456041:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df400, rc 0) 00000100:00100000:1.0:1787784899.456104:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030483c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33492:x1874627658319232:12345-192.168.203.43@tcp:101:bash.0 Request processed in 473us (661us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.456111:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1550 00000100:00100000:2.0:1787784899.467204:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.467298:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658319360 00000100:00100000:0.0:1787784899.467308:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658319360 00000100:00100000:0.0:1787784899.467365:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1551 00000100:00100000:0.0:1787784899.467386:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33493:x1874627658319360:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784899.467406:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.467426:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc824b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bcf expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467491:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467505:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467517:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467530:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467537:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467543:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467609:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467617:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467622:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467629:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.467633:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467640:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467652:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.467655:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.467659:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467664:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8252 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467675:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467680:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467686:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467692:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ea00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.467695:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467700:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467708:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff989520108a00) 00010000:00010000:0.0:1787784899.467711:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.467713:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467719:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8259 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467755:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc824b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bcf expref: 64 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467765:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.467768:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.467771:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc824b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bcf expref: 64 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.467795:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f800, rc 0) 00000100:00100000:0.0:1787784899.467858:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33493:x1874627658319360:12345-192.168.203.43@tcp:101:rm.0 Request processed in 484us (663us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.467867:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1551 00000100:00100000:3.0:1787784899.470322:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.470470:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658319488 00000100:00100000:1.0:1787784899.470480:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658319488 00000100:00100000:1.0:1787784899.470536:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 79 00000100:00100000:1.0:1787784899.470546:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+63:2354:x1874627658319488:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784899.470578:0:5817:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.470596:0:5817:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8260 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bd6 expref: -99 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470649:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470663:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470687:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470703:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470711:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470718:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470775:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470782:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470803:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470810:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.470814:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470821:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470832:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.470835:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.470839:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470847:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8267 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470859:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470865:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470871:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470876:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.470879:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470885:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470892:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x36fdbbdd (ffff98951b73b700) 00010000:00010000:1.0:1787784899.470895:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.470898:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470904:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc826e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470933:0:5817:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8260 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bd6 expref: 64 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470949:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.470952:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784899.470956:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8260 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bd6 expref: 64 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.470965:0:5817:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df400, rc -2) 00000100:00100000:1.0:1787784899.471029:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+63:2354:x1874627658319488:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 485us (716us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784899.471037:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 79 00000100:00100000:2.0:1787784899.475558:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.475687:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658319616 00000100:00100000:3.0:1787784899.475696:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658319616 00000100:00100000:3.0:1787784899.475754:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1552 00000100:00100000:3.0:1787784899.475764:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33493:x1874627658319616:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784899.475938:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.475971:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8275 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bdd expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.476043:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.476061:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.476072:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.476090:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.476098:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.476105:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476391:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476399:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476406:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476411:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.476416:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476423:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476435:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.476439:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.476444:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476450:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc827c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476464:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476469:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476475:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476481:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.476484:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476490:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476498:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff9895198b5900) 00010000:00010000:2.0:1787784899.476501:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.476504:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476510:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8283 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476543:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8275 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bdd expref: 64 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476555:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.476557:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.476561:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6200/0xb0ce63137acc8275 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bdd expref: 64 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.476571:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:2.0:1787784899.476634:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33493:x1874627658319616:12345-192.168.203.43@tcp:101:rm.0 Request processed in 871us (1085us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784899.476642:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1552 00000100:00100000:2.0:1787784899.492423:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.492513:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658319744 00000100:00100000:1.0:1787784899.492523:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658319744 00000100:00100000:1.0:1787784899.492583:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1553 00000100:00100000:1.0:1787784899.492593:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33494:x1874627658319744:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784899.492622:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.492642:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc828a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1be4 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492719:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492733:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492744:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492758:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492766:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492772:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492845:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492853:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492860:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492872:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.492876:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492884:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492895:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.492898:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.492902:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492909:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8291 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492923:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492929:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492934:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492939:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.492942:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492947:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492953:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff98951b73b700) 00010000:00010000:1.0:1787784899.492956:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.492958:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492963:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8298 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.492991:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc828a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1be4 expref: 64 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.493001:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.493003:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.493006:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc828a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1be4 expref: 64 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.493019:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df400, rc 0) 00000100:00100000:1.0:1787784899.493075:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33494:x1874627658319744:12345-192.168.203.43@tcp:101:cp.0 Request processed in 484us (661us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.493083:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1553 00000100:00100000:3.0:1787784899.496659:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.497123:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658319872 00000100:00100000:1.0:1787784899.497133:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658319872 00000100:00100000:1.0:1787784899.498537:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1554 00000100:00100000:1.0:1787784899.498549:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33494:x1874627658319872:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784899.498567:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.498585:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc829f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1beb expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498642:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498655:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498665:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498689:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498695:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498701:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498756:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498763:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498768:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498774:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.498778:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498983:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.498995:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.498997:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.499001:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499007:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc82a6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499021:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499026:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499032:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499037:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.499040:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499046:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499052:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff98951b73bb00) 00010000:00010000:1.0:1787784899.499054:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.499056:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499061:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc82ad lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499092:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc829f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1beb expref: 64 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499102:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.499104:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.499107:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc829f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1beb expref: 64 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.499116:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df600, rc 0) 00000100:00100000:1.0:1787784899.499193:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33494:x1874627658319872:12345-192.168.203.43@tcp:101:cp.0 Request processed in 647us (2542us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784899.499201:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1554 00000100:00100000:2.0:1787784899.502704:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.502900:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658320128 00000100:00100000:0.0:1787784899.502909:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658320128 00000100:00100000:0.0:1787784899.502966:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1555 00000100:00100000:0.0:1787784899.502975:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+63:33494:x1874627658320128:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.502993:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.503013:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc82b4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bf2 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.503133:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.503181:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.503191:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.503224:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.503237:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 113 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.503250:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668455168:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784899.503323:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fe40 going to sleep for 11 seconds 00010000:00010000:3.0:1787784899.505358:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784899.507593:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.507700:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658320256 00000100:00100000:2.0:1787784899.507710:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658320256 00010000:00010000:2.0:1787784899.507750:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 114 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.507800:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 450 00000100:00100000:2.0:1787784899.507810:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:2356:x1874627658320256:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784899.507824:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784899.507831:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 114 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.507844:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 114 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.507865:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.507872:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.507882:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.507889:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.507908:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784899.507965:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:2356:x1874627658320256:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 156us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784899.507973:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 450 00010000:00010000:0.0:1787784899.508469:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8228 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bba expref: 113 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.508487:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668455168:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784899.508518:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.508528:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.508534:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.508555:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.508565:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.508574:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784899.509120:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1f:0x0], index 0: 0 00010000:00010000:0.0:1787784899.509525:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509537:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509546:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509581:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509588:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509595:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509602:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ea00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.509606:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509613:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509624:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.509627:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.509631:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509638:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82bb lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784899.509648:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000090b2774c reply state 00000000fd72036e transno 4294968352 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.509657:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784899.509688:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc82b4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 66 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509702:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 65 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.509709:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:0.0:1787784899.509712:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784899.509715:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f000, rc 0) 00000100:00100000:0.0:1787784899.509973:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33494:x1874627658320128:12345-192.168.203.43@tcp:101:cp.0 Request processed in 6998us (7276us total) trans 4294968352 rc 0/0 00000100:00100000:0.0:1787784899.509982:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1555 00000100:00100000:3.0:1787784899.515639:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784899.515743:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.515755:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.515762:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.515767:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.515772:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.515783:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.515809:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff989520108a00) 00010000:00010000:1.0:1787784899.515812:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.515816:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.515822:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc82c2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.516337:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658320384 00000100:00100000:0.0:1787784899.516346:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658320384 00000100:00100000:0.0:1787784899.516484:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 457 00000100:00100000:0.0:1787784899.516495:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658320384:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.516512:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.516528:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1bf9 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.516552:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784899.516570:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1787784899.516574:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.516583:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.516597:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 11 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.516609:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704ee00, rc 0) 00000100:00100000:0.0:1787784899.516668:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658320384:12345-192.168.203.43@tcp:101:cp.0 Request processed in 176us (1038us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.516675:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 457 00000100:00100000:2.0:1787784899.519965:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.520038:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658320512 00000100:00100000:1.0:1787784899.520048:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658320512 00000100:00100000:1.0:1787784899.520099:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 458 00000100:00100000:1.0:1787784899.520107:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+11:33494:x1874627658320512:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784899.520121:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.520137:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c00 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.520201:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.520210:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784899.520214:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784899.520218:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.520247:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 12 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.520260:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 12 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.520274:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539250000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668455296:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784899.520345:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab46c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784899.523753:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.523916:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658320640 00000100:00100000:0.0:1787784899.523925:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658320640 00010000:00010000:0.0:1787784899.523961:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 13 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.524004:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:0.0:1787784899.524014:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658320640:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784899.524029:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784899.524049:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 13 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.524063:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 13 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.524087:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.524091:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.524100:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.524110:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.524145:0:5796:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 13 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.524195:0:5796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668455424:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784899.524257:0:5796:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fe40 going to sleep for 11 seconds 00010000:00010000:1.0:1787784899.524381:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b14a4357/0xb0ce63137acc82d7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf9 expref: 13 pid: 10688 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.524395:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539250000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668455296:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784899.524417:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.524431:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df600, rc 0) 00000100:00100000:1.0:1787784899.524458:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:33494:x1874627658320512:12345-192.168.203.43@tcp:101:cp.0 Request processed in 4352us (4503us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.524465:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 458 00000100:00100000:0.0:1787784899.526484:0:5796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5796:1874627668455424:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784899.526517:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784899.526561:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658320640:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2548us (2816us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.526570:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 451 00000100:00100000:3.0:1787784899.529755:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.529909:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658320896 00000100:00100000:1.0:1787784899.529918:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658320896 00000100:00100000:1.0:1787784899.529972:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 332 00000100:00100000:1.0:1787784899.529982:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+65:33494:x1874627658320896:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784899.530326:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33494:x1874627658320896:12345-192.168.203.43@tcp:35:cp.0 Request processed in 344us (577us total) trans 4294968353 rc 0/0 00000100:00100000:1.0:1787784899.530334:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 332 00000040:00080000:3.0:1787784899.569642:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784899.569651:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784899.569696:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 48 count 7 00000040:00080000:3.0:1787784899.569702:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:3.0:1787784899.570083:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668455552:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784899.570261:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784899.579608:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.579696:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668455552:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784899.579968:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658321024 00000100:00100000:1.0:1787784899.579975:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658321024 00000100:00100000:1.0:1787784899.580022:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1556 00000100:00100000:1.0:1787784899.580029:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33500:x1874627658321024:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784899.580056:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x30:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784899.580138:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33500:x1874627658321024:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 110us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.580144:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1556 00000100:00100000:2.0:1787784899.581252:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.581328:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658321152 00000100:00100000:1.0:1787784899.581335:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658321152 00000100:00100000:1.0:1787784899.581382:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1557 00000100:00100000:1.0:1787784899.581390:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33497:x1874627658321152:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784899.581416:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2e:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784899.581506:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33497:x1874627658321152:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 116us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.581514:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1557 00000100:00100000:2.0:1787784899.587641:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.587734:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658321280 00000100:00100000:0.0:1787784899.587745:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658321280 00000100:00100000:0.0:1787784899.587816:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1558 00000100:00100000:0.0:1787784899.587828:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33499:x1874627658321280:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784899.587871:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2f:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784899.587983:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+66:33499:x1874627658321280:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 157us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.587991:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1558 00000100:00100000:3.0:1787784899.601182:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.601278:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322176 00000100:00100000:0.0:1787784899.601288:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322176 00000100:00100000:0.0:1787784899.601350:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1559 00000100:00100000:0.0:1787784899.601360:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33498:x1874627658322176:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784899.601392:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.601414:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc82e5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c1c expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601486:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601506:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601518:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601534:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601541:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601548:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601647:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f600/0xb0ce63137acc82fa lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601656:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f600/0xb0ce63137acc82fa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601664:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f600/0xb0ce63137acc82fa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784899.601696:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x32:0x0]: returning size 158 00000004:00200000:0.0:1787784899.601701:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784899.601718:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f600/0xb0ce63137acc82fa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601731:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f600/0xb0ce63137acc82fa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601743:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601750:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601756:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601761:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ea00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.601765:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.601773:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602050:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.602055:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.602060:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602067:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc82ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602080:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602087:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602092:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602098:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.602100:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602106:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602113:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff989520108a00) 00010000:00010000:0.0:1787784899.602116:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.602119:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602124:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc82f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602460:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc82e5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c1c expref: 115 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602476:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f600/0xb0ce63137acc82fa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c1c expref: 114 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.602484:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:0.0:1787784899.602486:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784899.602489:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f600, rc 0) 00000100:00100000:0.0:1787784899.602605:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33498:x1874627658322176:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1246us (1468us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.602614:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1559 00000100:00100000:2.0:1787784899.604622:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.604648:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.604759:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.606068:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658321536 00000100:00100000:0.0:1787784899.606080:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658321536 00000100:00100000:0.0:1787784899.606137:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658321792 00000100:00100000:0.0:1787784899.606140:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658321792 00000100:00100000:0.0:1787784899.606185:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322048 00000100:00100000:0.0:1787784899.606189:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322048 00000100:00100000:0.0:1787784899.606211:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1560 00000100:00100000:0.0:1787784899.606220:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+68:33497:x1874627658321536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784899.606241:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.606261:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8301 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c07 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.607226:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1561 00000100:00100000:2.0:1787784899.607243:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33500:x1874627658321792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784899.607255:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.607268:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8308 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c0e expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607349:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc830f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607364:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc830f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607388:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc830f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784899.607399:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x30:0x0]: returning size 158 00010000:00010000:2.0:1787784899.607526:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8308 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c0e expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607544:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc830f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c0e expref: 71 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607552:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:2.0:1787784899.607555:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784899.607559:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111000, rc 0) 00000100:00100000:2.0:1787784899.607615:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+71:33500:x1874627658321792:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 378us (2965us total) trans 4294968354 rc 0/0 00000100:00100000:2.0:1787784899.607623:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1561 00000100:00100000:2.0:1787784899.607648:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1562 00000100:00100000:2.0:1787784899.607656:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33499:x1874627658322048:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784899.607664:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.607683:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc831d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c15 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607728:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8324 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607736:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8324 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607749:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8324 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784899.607756:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2f:0x0]: returning size 158 00010000:00010000:2.0:1787784899.607973:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc831d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c15 expref: 73 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607986:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8324 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c15 expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.607994:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:2.0:1787784899.607997:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784899.607999:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110200, rc 0) 00000100:00100000:2.0:1787784899.608022:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33499:x1874627658322048:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 367us (3263us total) trans 4294968355 rc 0/0 00000100:00100000:2.0:1787784899.608028:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1562 00010000:00010000:0.0:1787784899.608139:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8332 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.608174:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8332 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.608183:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8332 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784899.608191:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2e:0x0]: returning size 158 00010000:00010000:0.0:1787784899.608267:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8301 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c07 expref: 73 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.608281:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8332 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c07 expref: 72 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.608288:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:0.0:1787784899.608291:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784899.608294:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fe00, rc 0) 00000100:00100000:0.0:1787784899.608322:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33497:x1874627658321536:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2103us (3708us total) trans 4294968356 rc 0/0 00000100:00100000:0.0:1787784899.608328:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1560 00000100:00100000:2.0:1787784899.610651:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.610711:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322304 00000100:00100000:3.0:1787784899.610719:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322304 00000100:00100000:3.0:1787784899.610760:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:3.0:1787784899.610769:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658322304:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784899.610807:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.610824:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8340 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c23 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.610876:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668455680:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784899.610955:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169e40 going to sleep for 11 seconds 00002000:00010000:3.0:1787784899.613987:0:6671:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1787784899.613997:0:6671:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb mtime: 0 -> 1787784898 00002000:00010000:3.0:1787784899.613999:0:6671:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb atime: 0 -> 1787784898 00002000:00010000:3.0:1787784899.614002:0:6671:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb ctime: 0 -> 1787784898 00000100:00100000:3.0:1787784899.614024:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989518f0b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668455680:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784899.614058:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8340 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c23 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.614081:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8340 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c23 expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.614094:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784899.614176:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2355:x1874627658322304:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 3378us (3502us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784899.614185:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:3.0:1787784899.616397:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.616439:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.616464:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.616488:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.616512:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322816 00000100:00100000:3.0:1787784899.616520:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322816 00000100:00100000:3.0:1787784899.616570:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 68 00000100:00100000:3.0:1787784899.616579:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+72:2354:x1874627658322816:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784899.616593:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784899.616614:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc830f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c0e expref: 72 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.616630:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:3.0:1787784899.616635:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784899.616672:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+71:2354:x1874627658322816:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 94us (182us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.616679:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 68 00000100:00100000:1.0:1787784899.616802:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322560 00000100:00100000:1.0:1787784899.616807:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322560 00000100:00100000:1.0:1787784899.616843:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 80 00000100:00100000:1.0:1787784899.616851:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:2357:x1874627658322560:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784899.616893:0:13404:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.616909:0:13404:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8347 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c2a expref: -99 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.616960:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.616972:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.616983:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.616996:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617003:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617010:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617068:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617076:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617081:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617087:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.617091:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617098:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617107:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.617110:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.617113:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617120:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc834e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617129:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617134:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617140:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617170:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.617173:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617179:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617186:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff98951b73bb00) 00010000:00010000:1.0:1787784899.617188:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.617191:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617197:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8355 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617221:0:13404:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8347 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c2a expref: 115 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617229:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.617232:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784899.617235:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8347 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c2a expref: 115 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.617245:0:13404:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dee00, rc -2) 00000100:00100000:1.0:1787784899.617278:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:2357:x1874627658322560:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 427us (888us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784899.617285:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 80 00000100:00100000:0.0:1787784899.618055:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322432 00000100:00100000:0.0:1787784899.618063:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322432 00000100:00100000:0.0:1787784899.618101:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322688 00000100:00100000:0.0:1787784899.618104:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322688 00000100:00100000:0.0:1787784899.618134:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:0.0:1787784899.618143:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658322432:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784899.618208:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618225:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618233:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.618237:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.618241:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618269:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 12 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618281:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 12 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.618293:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668455808:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784899.618347:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f900 going to sleep for 11 seconds 00000100:00100000:0.0:1787784899.618373:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 461 00000100:00100000:0.0:1787784899.618381:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658322688:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784899.618396:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618405:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618411:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1d:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784899.618413:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.618416:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618432:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 12 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.618439:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 12 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.618448:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668455936:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784899.618467:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fe40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784899.623537:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.623614:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658322944 00000100:00100000:3.0:1787784899.623623:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658322944 00000100:00100000:3.0:1787784899.623736:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 69 00000100:00100000:3.0:1787784899.623746:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+71:2356:x1874627658322944:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784899.623763:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784899.623799:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8332 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c07 expref: 71 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.623819:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:3.0:1787784899.623824:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784899.623880:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+70:2356:x1874627658322944:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 135us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.623887:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 69 00000100:00100000:0.0:1787784899.631692:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.631720:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.631747:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323072 00000100:00100000:0.0:1787784899.631756:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323072 00000100:00100000:1.0:1787784899.632099:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668455936:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784899.632135:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:0.0:1787784899.632330:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1d:0x0] (0->157) req@ffff98940304a580 x1874627658323072/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.632339:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304a580 with x1874627658323072 ext(0->157) 00010000:00010000:0.0:1787784899.632346:0:18437:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 13 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.632356:0:18437:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 13 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.632362:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff98940304a580 00000100:00100000:0.0:1787784899.632388:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323200 00000100:00100000:0.0:1787784899.632391:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323200 00002000:00100000:0.0:1787784899.632407:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1e:0x0] (0->157) req@ffff989403048000 x1874627658323200/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.632415:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403048000 with x1874627658323200 ext(0->157) 00010000:00010000:0.0:1787784899.632422:0:18437:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 14 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.632433:0:18437:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 14 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.632440:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989403048000 00000100:00100000:0.0:1787784899.632455:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 17 00000100:00100000:0.0:1787784899.632465:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658323072:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784899.632919:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 18 00000100:00100000:1.0:1787784899.632929:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2355:x1874627658323200:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784899.633333:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2355:x1874627658323200:12345-192.168.203.43@tcp:4:cp.0 Request processed in 404us (1610us total) trans 4294967376 rc 0/0 00000100:00100000:1.0:1787784899.633342:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 18 00002000:00100000:1.0:1787784899.633351:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1e:0x0] (0->157) req@ffff989403048000 x1874627658323200/t4294967376(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/448 e 0 to 0 dl 1787784910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784899.633359:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989403048000 with x1874627658323200 ext(0->157) 00010000:00010000:1.0:1787784899.633380:0:6676:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 16 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.633391:0:6676:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 16 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784899.633399:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989403048000 00000100:00100000:0.0:1787784899.634103:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658323072:12345-192.168.203.43@tcp:4:cp.0 Request processed in 1635us (2416us total) trans 4294967377 rc 0/0 00000100:00100000:0.0:1787784899.634115:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 17 00002000:00100000:0.0:1787784899.634123:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1d:0x0] (0->157) req@ffff98940304a580 x1874627658323072/t4294967377(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/448 e 0 to 0 dl 1787784910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.634132:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98940304a580 with x1874627658323072 ext(0->157) 00010000:00010000:0.0:1787784899.634138:0:18437:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 17 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.634176:0:18437:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 17 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.634183:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff98940304a580 00000100:00100000:0.0:1787784899.634352:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.634755:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323328 00000100:00100000:3.0:1787784899.634763:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323328 00000100:00100000:3.0:1787784899.634931:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 462 00000100:00100000:3.0:1787784899.634941:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658323328:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.634979:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.634995:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.635002:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.635008:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784899.635012:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 4/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.635044:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 12 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.635055:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 12 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.635067:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6ab840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668456064:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784899.635118:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784899.635217:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403049e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668455808:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784899.635248:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.635886:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.636089:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323456 00000100:00100000:1.0:1787784899.636096:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323456 00000100:00100000:1.0:1787784899.636143:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1563 00000100:00100000:1.0:1787784899.636175:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33498:x1874627658323456:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784899.636191:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.636219:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8371 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c62 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.636282:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.636296:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.636306:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784899.636357:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:1.0:1787784899.636361:0:5813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 5 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784899.636395:0:5813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.636421:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8371 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 116 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.636438:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 115 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.636446:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784899.636449:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787784899.636452:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00000100:00100000:1.0:1787784899.636498:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33498:x1874627658323456:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 329us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.636505:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1563 00000100:00100000:0.0:1787784899.640318:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.640854:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323584 00000100:00100000:1.0:1787784899.640865:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323584 00000100:00100000:1.0:1787784899.640934:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 70 00000100:00100000:1.0:1787784899.640945:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+70:2357:x1874627658323584:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784899.640961:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784899.640986:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8324 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c15 expref: 70 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.641005:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:1.0:1787784899.641009:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784899.641316:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+69:2357:x1874627658323584:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 372us (1016us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.641325:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 70 00000100:00100000:2.0:1787784899.643708:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.643802:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.643838:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.643876:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323968 00000100:00100000:1.0:1787784899.643883:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323968 00002000:00100000:1.0:1787784899.643927:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x20:0x0] (0->157) req@ffff989530517480 x1874627658323968/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784899.643937:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530517480 with x1874627658323968 ext(0->157) 00010000:00010000:1.0:1787784899.643946:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 13 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.643959:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 13 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784899.643966:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989530517480 00000100:00100000:1.0:1787784899.643991:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 19 00000100:00100000:1.0:1787784899.644001:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2354:x1874627658323968:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784899.644523:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2354:x1874627658323968:12345-192.168.203.43@tcp:4:cp.0 Request processed in 520us (718us total) trans 4294967391 rc 0/0 00000100:00100000:1.0:1787784899.644532:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 19 00002000:00100000:1.0:1787784899.644539:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x20:0x0] (0->157) req@ffff989530517480 x1874627658323968/t4294967391(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/448 e 0 to 0 dl 1787784910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784899.644548:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530517480 with x1874627658323968 ext(0->157) 00010000:00010000:1.0:1787784899.644554:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 15 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.644564:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 15 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784899.644571:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989530517480 00000100:00100000:3.0:1787784899.644687:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6ab840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668456064:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784899.644710:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.648538:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.653740:0:13825:0:(service.c:3462:ptlrpc_start_thread()) ost[0] started 5 min 3 max 116 00000100:00100000:3.0:1787784899.653760:0:13825:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost00_005' 00000100:00100000:3.0:1787784899.654339:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658324096 00000100:00100000:3.0:1787784899.654349:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658324096 00000100:00100000:2.0:1787784899.654511:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 463 00000100:00100000:2.0:1787784899.654522:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658324096:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784899.654540:0:22706:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.654556:0:22706:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc837f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c69 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.654622:0:22706:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:22706:1874627668456192:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784899.654708:0:22706:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088fd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784899.656073:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323712 00000100:00100000:0.0:1787784899.656084:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323712 00010000:00010000:0.0:1787784899.656116:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 17 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.656143:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658323840 00000100:00100000:0.0:1787784899.656159:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658323840 00010000:00010000:0.0:1787784899.656172:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 18 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.656196:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:0.0:1787784899.656215:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2357:x1874627658323712:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784899.656226:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784899.656244:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 18 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656257:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 18 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656278:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea33755a/0xb0ce63137acc7f2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19fa expref: 18 pid: 10688 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656292:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.656296:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.656302:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656311:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656347:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784899.656495:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658323712:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 279us (12792us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.656503:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 452 00000100:00100000:0.0:1787784899.656527:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 453 00000100:00100000:0.0:1787784899.656535:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c97050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2354:x1874627658323840:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784899.656543:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784899.656554:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 16 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656566:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 16 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656579:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc80d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa2 expref: 16 pid: 6669 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656590:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.656593:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.656598:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656606:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656628:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784899.656648:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c97050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2354:x1874627658323840:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 114us (12809us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.656654:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 453 00010000:00010000:0.0:1787784899.656706:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656716:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656743:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656750:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656757:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656763:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fc00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784899.656767:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656774:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656856:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.656865:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db4baa8e/0xb0ce63137acc835c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.656911:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658322432:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 38769us (40471us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.656922:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:0.0:1787784899.656978:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.657003:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658324224 00000100:00100000:0.0:1787784899.657007:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658324224 00010000:00010000:0.0:1787784899.657021:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 15 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.657040:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 454 00000100:00100000:0.0:1787784899.657046:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658324224:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784899.657054:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784899.657067:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 15 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.657078:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 15 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.657093:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f26197df/0xb0ce63137acc7fff lrc: 0/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a4e expref: 15 pid: 13825 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.657104:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.657107:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.657111:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.657118:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.657137:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784899.657190:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658324224:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 143us (210us total) trans 0 rc 0/0 00010000:00010000:3.0:1787784899.657298:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657307:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657324:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657331:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657339:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657345:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784899.657348:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657355:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657366:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.657387:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fefb50b3/0xb0ce63137acc836a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.657427:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2354:x1874627658323328:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 22488us (23076us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.657434:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 462 00000100:00100000:0.0:1787784899.658045:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 454 00010000:00010000:1.0:1787784899.658600:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658614:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658637:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658644:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658652:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658664:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ea00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784899.658668:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658677:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658692:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.658700:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc8363 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.658756:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658322688:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 40372us (42288us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.658764:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 461 00000100:00100000:0.0:1787784899.660970:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.665166:0:22706:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530514000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:22706:1874627668456192:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784899.665222:0:22706:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc837f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c69 expref: 17 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.665248:0:22706:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d45f284/0xb0ce63137acc837f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c69 expref: 17 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.665262:0:22706:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111e00, rc 0) 00000100:00100000:2.0:1787784899.665326:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658324096:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 10805us (16795us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784899.665336:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 463 00000100:00100000:0.0:1787784899.666607:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.666724:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.671919:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658324480 00000100:00100000:1.0:1787784899.671933:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658324480 00000100:00100000:1.0:1787784899.672004:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658324736 00000100:00100000:1.0:1787784899.672008:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658324736 00000100:00100000:1.0:1787784899.672041:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1564 00000100:00100000:1.0:1787784899.672050:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33500:x1874627658324480:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784899.672191:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a91e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33500:x1874627658324480:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 141us (11228us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.672209:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1564 00000100:00100000:1.0:1787784899.672228:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1565 00000100:00100000:1.0:1787784899.672236:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33497:x1874627658324736:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784899.672270:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33497:x1874627658324736:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 35us (5668us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.672277:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1565 00000100:00100000:2.0:1787784899.673238:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658324992 00000100:00100000:2.0:1787784899.673246:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658324992 00000100:00100000:2.0:1787784899.673290:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1566 00000100:00100000:2.0:1787784899.673299:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33499:x1874627658324992:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784899.673356:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33499:x1874627658324992:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 59us (6632us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784899.673367:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1566 00000100:00100000:0.0:1787784899.687036:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.687087:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.687177:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658325248 00000100:00100000:1.0:1787784899.687187:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658325248 00000100:00100000:1.0:1787784899.687255:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1567 00000100:00100000:1.0:1787784899.687266:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33498:x1874627658325248:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:1.0:1787784899.687504:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].49, 136 off14856 00000100:00100000:1.0:1787784899.687593:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33498:x1874627658325248:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 329us (567us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.687601:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1567 00000100:00100000:3.0:1787784899.688427:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658325376 00000100:00100000:3.0:1787784899.688434:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658325376 00000100:00100000:3.0:1787784899.688478:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 464 00000100:00100000:3.0:1787784899.688486:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2357:x1874627658325376:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.688499:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.688516:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c70 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784899.688539:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1e:0x0] initial LVB size: 158, mtime: 0x6a8f6ec2, atime: 0x0, ctime: 0x6a8f6ec2, blocks: 0x1 00010000:00010000:3.0:1787784899.688554:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1787784899.688559:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.688568:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 14 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.688583:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 14 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.688596:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6200, rc 0) 00000100:00100000:3.0:1787784899.688631:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658325376:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 145us (1544us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.688637:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 464 00000100:00100000:3.0:1787784899.688677:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.688742:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658325504 00000100:00100000:3.0:1787784899.688745:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658325504 00000100:00100000:3.0:1787784899.688767:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 465 00000100:00100000:3.0:1787784899.688773:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658325504:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.688780:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.688800:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c77 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784899.688813:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1d:0x0] initial LVB size: 158, mtime: 0x6a8f6ec2, atime: 0x0, ctime: 0x6a8f6ec2, blocks: 0x1 00010000:00010000:3.0:1787784899.688822:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1787784899.688825:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.688831:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 15 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.688840:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 15 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.688849:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6c00, rc 0) 00000100:00100000:3.0:1787784899.688872:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2356:x1874627658325504:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 99us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.688879:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 465 00000100:00100000:3.0:1787784899.698361:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.698596:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658325632 00000100:00100000:1.0:1787784899.698604:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658325632 00000100:00100000:1.0:1787784899.698656:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:1.0:1787784899.698667:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2356:x1874627658325632:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784899.698686:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.698707:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c7e expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784899.698734:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x20:0x0] initial LVB size: 158, mtime: 0x6a8f6ec2, atime: 0x0, ctime: 0x6a8f6ec2, blocks: 0x1 00010000:00010000:1.0:1787784899.698755:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1787784899.698760:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784899.698770:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 14 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.698860:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 14 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.698874:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dee00, rc 0) 00000100:00100000:1.0:1787784899.698965:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2356:x1874627658325632:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 301us (613us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.698972:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:0.0:1787784899.724840:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.724888:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.724922:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.724948:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.725008:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658325760 00000100:00100000:0.0:1787784899.725019:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658325760 00000100:00100000:0.0:1787784899.725073:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658326400 00000100:00100000:0.0:1787784899.725076:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658326400 00000100:00100000:0.0:1787784899.725098:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658326144 00000100:00100000:0.0:1787784899.725101:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658326144 00000100:00100000:0.0:1787784899.725119:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1568 00000100:00100000:0.0:1787784899.725130:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33504:x1874627658325760:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:0.0:1787784899.725175:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.725203:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc839b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c85 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725292:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725311:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725325:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725343:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725350:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725355:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725444:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725450:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725457:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725463:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.725467:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725475:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725487:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.725491:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.725497:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725502:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83a2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725519:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725524:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725530:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725534:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.725536:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725541:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725546:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff989520108b00) 00010000:00010000:0.0:1787784899.725548:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.725551:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725555:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83a9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725594:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc839b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c85 expref: 116 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725605:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.725607:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.725610:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc839b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c85 expref: 116 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.725617:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f200, rc 0) 00000100:00100000:0.0:1787784899.725676:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33504:x1874627658325760:12345-192.168.203.43@tcp:101:bash.0 Request processed in 549us (848us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.725684:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1568 00000100:00100000:0.0:1787784899.725712:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1569 00000100:00100000:0.0:1787784899.725719:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33500:x1874627658326400:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784899.725758:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33500:x1874627658326400:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 41us (836us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.725764:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1569 00000100:00100000:0.0:1787784899.725777:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1570 00000100:00100000:0.0:1787784899.725795:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33497:x1874627658326144:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784899.725829:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33497:x1874627658326144:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 45us (880us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.725835:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1570 00000100:00100000:3.0:1787784899.726388:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658325888 00000100:00100000:3.0:1787784899.726398:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658325888 00000100:00100000:3.0:1787784899.726447:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 81 00000100:00100000:3.0:1787784899.726456:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:2355:x1874627658325888:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:3.0:1787784899.726485:0:5816:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.726505:0:5816:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c8c expref: -99 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726549:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726562:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726571:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726586:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726593:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726599:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726650:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726658:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726664:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726670:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.726674:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726681:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726691:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.726695:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.726699:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726705:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc83b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726718:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726724:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726729:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726734:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.726739:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726745:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726751:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x4b66ad81 (ffff9895198b5100) 00010000:00010000:3.0:1787784899.726754:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.726757:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726763:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83be lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726869:0:5816:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83b0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c8c expref: 116 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726880:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.726882:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784899.726886:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c8c expref: 116 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.726896:0:5816:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc -2) 00000100:00100000:3.0:1787784899.726942:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:2355:x1874627658325888:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 488us (2054us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784899.726950:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 81 00000100:00100000:3.0:1787784899.727016:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.727184:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658326656 00000100:00100000:3.0:1787784899.727189:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658326656 00000100:00100000:3.0:1787784899.727214:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1571 00000100:00100000:3.0:1787784899.727222:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33499:x1874627658326656:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784899.727269:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33499:x1874627658326656:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 47us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.727276:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1571 00000100:00100000:3.0:1787784899.734880:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.734924:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.735012:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658326784 00000100:00100000:0.0:1787784899.735023:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658326784 00002000:00100000:0.0:1787784899.735075:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1d:0x0] (0->4095) req@ffff98953f6aa580 x1874627658326784/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 488/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.735095:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f6aa580 with x1874627658326784 ext(0->4095) 00002000:00010000:0.0:1787784899.735101:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98953f6aa580 00000100:00100000:0.0:1787784899.735127:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658326912 00000100:00100000:0.0:1787784899.735130:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658326912 00002000:00100000:0.0:1787784899.735176:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1e:0x0] (0->4095) req@ffff98953f6a8000 x1874627658326912/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 488/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.735186:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f6a8000 with x1874627658326912 ext(0->4095) 00002000:00010000:0.0:1787784899.735189:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98953f6a8000 00000100:00100000:0.0:1787784899.735208:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 125 00000100:00100000:0.0:1787784899.735218:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aa580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2357:x1874627658326784:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1787784899.735426:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aa580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2357:x1874627658326784:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 208us (555us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784899.735434:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 125 00002000:00100000:0.0:1787784899.735440:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1d:0x0] (0->4095) req@ffff98953f6aa580 x1874627658326784/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 488/4536 e 0 to 0 dl 1787784910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.735447:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f6aa580 with x1874627658326784 ext(0->4095) 00002000:00010000:0.0:1787784899.735449:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98953f6aa580 00000100:00100000:0.0:1787784899.735468:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 126 00000100:00100000:0.0:1787784899.735473:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658326912:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1787784899.735530:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658326912:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 56us (606us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787784899.735535:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 126 00002000:00100000:0.0:1787784899.735540:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1e:0x0] (0->4095) req@ffff98953f6a8000 x1874627658326912/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 488/4536 e 0 to 0 dl 1787784910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.735546:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f6a8000 with x1874627658326912 ext(0->4095) 00002000:00010000:0.0:1787784899.735548:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98953f6a8000 00000100:00100000:2.0:1787784899.750197:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.750290:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.750520:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327040 00000100:00100000:1.0:1787784899.750531:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327040 00002000:00100000:1.0:1787784899.750582:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x20:0x0] (0->4095) req@ffff989530514000 x1874627658327040/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 488/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784899.750594:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530514000 with x1874627658327040 ext(0->4095) 00002000:00010000:1.0:1787784899.750600:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530514000 00000100:00100000:1.0:1787784899.750637:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 127 00000100:00100000:1.0:1787784899.750647:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658327040:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1787784899.750860:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658327040:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 214us (673us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787784899.750868:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 127 00002000:00100000:1.0:1787784899.750875:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x20:0x0] (0->4095) req@ffff989530514000 x1874627658327040/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 488/4536 e 0 to 0 dl 1787784910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784899.750884:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989530514000 with x1874627658327040 ext(0->4095) 00002000:00010000:1.0:1787784899.750887:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989530514000 00000100:00100000:0.0:1787784899.751275:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327168 00000100:00100000:0.0:1787784899.751281:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327168 00000100:00100000:0.0:1787784899.751323:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1572 00000100:00100000:0.0:1787784899.751332:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33505:x1874627658327168:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784899.751344:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.751364:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc83c5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c93 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751427:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751443:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751454:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751469:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751476:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751482:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751544:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751553:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751560:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751566:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.751570:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751576:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751588:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.751591:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.751595:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751601:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc83cc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751613:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751619:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751624:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751630:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.751633:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751639:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751646:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff989520108b00) 00010000:00010000:0.0:1787784899.751649:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.751652:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751659:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc83d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751693:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc83c5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c93 expref: 70 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751703:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.751705:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.751708:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc83c5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c93 expref: 70 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.751717:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f200, rc 0) 00000100:00100000:0.0:1787784899.751752:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33505:x1874627658327168:12345-192.168.203.43@tcp:101:rm.0 Request processed in 422us (1462us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.751760:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1572 00000100:00100000:2.0:1787784899.753747:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.753847:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.753930:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327424 00000100:00100000:1.0:1787784899.753940:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327424 00000100:00100000:1.0:1787784899.753995:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1573 00000100:00100000:1.0:1787784899.754004:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+69:33497:x1874627658327424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784899.754023:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.754042:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc83da lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ca1 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.754074:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.754088:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.754100:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.754228:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfa00/0xb0ce63137acc83da lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 71 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.754244:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 70 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.754253:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:1.0:1787784899.754256:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784899.754259:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de400, rc 0) 00000100:00100000:1.0:1787784899.754308:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305143c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33497:x1874627658327424:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 306us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.754315:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1573 00000100:00100000:3.0:1787784899.754716:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327296 00000100:00100000:3.0:1787784899.754721:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327296 00000100:00100000:3.0:1787784899.754757:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 82 00000100:00100000:3.0:1787784899.754764:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+70:2357:x1874627658327296:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:3.0:1787784899.754798:0:5817:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.754810:0:5817:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc83e8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1c9a expref: -99 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754845:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754855:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754864:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754876:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754883:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754894:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754950:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754957:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754963:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754970:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.754974:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754982:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.754991:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.755062:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.755066:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755074:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc83ef lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755084:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755090:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755097:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755103:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.755106:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755111:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755119:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x36fdbbdd (ffff9895198b5100) 00010000:00010000:3.0:1787784899.755121:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784899.755124:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755129:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc83f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755177:0:5817:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc83e8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c9a expref: 71 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755188:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784899.755190:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784899.755194:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc83e8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c9a expref: 71 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.755205:0:5817:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc -2) 00000100:00100000:3.0:1787784899.755252:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+70:2357:x1874627658327296:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 488us (1513us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784899.755264:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 82 00000100:00100000:2.0:1787784899.756686:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.757641:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327552 00000100:00100000:0.0:1787784899.757651:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327552 00000100:00100000:0.0:1787784899.757701:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1574 00000100:00100000:0.0:1787784899.757709:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33505:x1874627658327552:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784899.757725:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.757743:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc83fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ca8 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757811:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757824:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757834:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757848:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757855:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757862:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757921:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757928:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757934:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757940:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.757944:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757950:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757962:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.757964:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.757968:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757974:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8404 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757986:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757991:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.757996:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.758002:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.758004:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.758009:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.758019:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff989520108100) 00010000:00010000:0.0:1787784899.758021:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.758023:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.758028:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc840b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.758058:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc83fd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca8 expref: 71 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.758066:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.758068:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.758071:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc83fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca8 expref: 71 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.758079:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fe00, rc 0) 00000100:00100000:0.0:1787784899.758128:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33505:x1874627658327552:12345-192.168.203.43@tcp:101:rm.0 Request processed in 419us (1446us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.758135:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1574 00000100:00100000:1.0:1787784899.765234:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.766212:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327680 00000100:00100000:3.0:1787784899.766224:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327680 00000100:00100000:3.0:1787784899.766283:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1575 00000100:00100000:3.0:1787784899.766294:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+70:33500:x1874627658327680:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.766314:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.766334:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8412 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1caf expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.766658:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.766673:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.766685:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.766798:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8412 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 72 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.766817:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 71 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.766826:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:3.0:1787784899.766829:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784899.766833:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784899.766938:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539250000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+71:33500:x1874627658327680:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 646us (1713us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.766947:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1575 00000100:00100000:2.0:1787784899.770224:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.770264:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327808 00000100:00100000:0.0:1787784899.770274:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327808 00000100:00100000:2.0:1787784899.770290:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.770328:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658327936 00000100:00100000:0.0:1787784899.770333:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658327936 00000100:00100000:0.0:1787784899.770363:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1576 00000100:00100000:0.0:1787784899.770480:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33497:x1874627658327808:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784899.770522:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2e:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784899.770633:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530516580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33497:x1874627658327808:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 158us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.771241:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1577 00000100:00100000:3.0:1787784899.771254:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33499:x1874627658327936:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.771267:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.771296:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc8420 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cb6 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.771350:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.771363:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.771398:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1576 00010000:00010000:3.0:1787784899.771454:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.771555:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc8420 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 73 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.771574:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 72 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.771583:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:3.0:1787784899.771586:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784899.771589:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784899.771620:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33499:x1874627658327936:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 372us (1331us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.771631:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1577 00000100:00100000:2.0:1787784899.773981:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.774063:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658328064 00000100:00100000:3.0:1787784899.774072:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658328064 00000100:00100000:3.0:1787784899.774121:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1578 00000100:00100000:3.0:1787784899.774129:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33500:x1874627658328064:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784899.774186:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x30:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784899.774311:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530515a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33500:x1874627658328064:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 183us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.774319:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1578 00000100:00100000:2.0:1787784899.778521:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.778888:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658328192 00000100:00100000:3.0:1787784899.778897:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658328192 00000100:00100000:3.0:1787784899.778962:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 467 00000100:00100000:3.0:1787784899.778972:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658328192:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.779015:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779033:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.779037:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.779043:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779288:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779321:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779332:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779340:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779347:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784899.779351:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779360:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779375:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.779383:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc842e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.779464:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658328192:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 493us (952us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.779473:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 467 00000100:00100000:1.0:1787784899.779728:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.780848:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658328320 00000100:00100000:3.0:1787784899.780859:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658328320 00000100:00100000:3.0:1787784899.780913:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 468 00000100:00100000:3.0:1787784899.780924:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658328320:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.780959:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.780975:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.780979:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784899.780989:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781001:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781034:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781043:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781052:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781059:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784899.781063:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781072:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781086:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.781096:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8435 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.781205:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658328320:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 282us (1477us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.781214:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 468 00000100:00100000:1.0:1787784899.786206:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.786376:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.786560:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.786622:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658328448 00000100:00100000:0.0:1787784899.786631:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658328448 00000100:00100000:0.0:1787784899.786687:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658328704 00000100:00100000:0.0:1787784899.786691:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658328704 00000100:00100000:0.0:1787784899.786718:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1579 00000100:00100000:0.0:1787784899.786729:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+73:33499:x1874627658328448:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000100:00100000:3.0:1787784899.786733:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1580 00000100:00100000:3.0:1787784899.786746:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+73:33497:x1874627658328704:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000004:00000080:0.0:1787784899.786775:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2f:0x0] from 18446744073709551615 #0 00010000:00010000:3.0:1787784899.786831:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.786851:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 73 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.786861:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.786894:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 73 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.786908:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 73 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.786928:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668456320:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784899.787325:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+73:33499:x1874627658328448:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 595us (1125us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.787334:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1579 00000100:00100000:1.0:1787784899.787672:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658328960 00000100:00100000:1.0:1787784899.787678:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658328960 00000100:00100000:1.0:1787784899.787712:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1581 00000100:00100000:1.0:1787784899.787720:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+73:33500:x1874627658328960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784899.787760:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.787775:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 73 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.787794:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.787816:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 73 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.787827:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 73 pid: 5812 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.787837:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668456448:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784899.787889:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784899.796855:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.797931:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784899.800573:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658329088 00000100:00100000:0.0:1787784899.800585:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658329088 00000100:00100000:0.0:1787784899.800641:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1582 00000100:00100000:0.0:1787784899.800651:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+74:33506:x1874627658329088:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.800671:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.800691:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc844a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cbd expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800749:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800762:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800773:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800799:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800807:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800814:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800871:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800879:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800886:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800891:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.800895:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800902:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800912:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.800915:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.800919:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800926:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8451 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800938:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800944:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800949:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800955:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784899.800958:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800963:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800970:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff989520108100) 00010000:00010000:0.0:1787784899.800972:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.800975:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.800980:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8458 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.801010:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc844a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cbd expref: 75 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.801019:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.801021:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784899.801025:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc844a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cbd expref: 75 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.801034:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fe00, rc 0) 00000100:00100000:0.0:1787784899.801098:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+74:33506:x1874627658329088:12345-192.168.203.43@tcp:101:cp.0 Request processed in 450us (4252us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784899.801106:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1582 00000100:00100000:1.0:1787784899.802540:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.802568:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.802588:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.802613:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658329472 00000100:00100000:1.0:1787784899.802619:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658329472 00010000:00010000:1.0:1787784899.802646:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 74 pid: 5812 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.802667:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658329344 00000100:00100000:1.0:1787784899.802669:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658329344 00010000:00010000:1.0:1787784899.802680:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 75 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.802835:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 455 00000100:00100000:1.0:1787784899.802844:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+75:2357:x1874627658329472:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784899.802857:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784899.802863:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 75 pid: 5812 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.802872:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 75 pid: 5812 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.802888:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:1.0:1787784899.802893:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.802901:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.802908:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.802927:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784899.802957:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+75:2357:x1874627658329472:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 116us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.802965:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 455 00010000:00010000:3.0:1787784899.802979:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8419 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1caf expref: 74 pid: 5812 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.802983:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 456 00000100:00100000:3.0:1787784899.803003:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668456448:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784899.803022:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803031:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803037:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.803115:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+73:2357:x1874627658329344:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784899.803128:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784899.803134:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 73 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.803144:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 73 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.803179:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:1.0:1787784899.803184:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.803192:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.803198:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.803210:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784899.803229:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+74:2357:x1874627658329344:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 238us (641us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.803236:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 456 00010000:00010000:3.0:1787784899.803242:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803250:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803257:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803264:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784899.803269:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803281:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803291:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:3.0:1787784899.803293:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784899.803297:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.803303:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8443 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.803409:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+73:33500:x1874627658328960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 15688us (16854us total) trans 4294968357 rc 0/0 00000100:00100000:3.0:1787784899.803418:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1581 00010000:00010000:2.0:1787784899.803593:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc83e1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ca1 expref: 72 pid: 20438 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.803607:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668456320:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784899.803620:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803628:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803633:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803713:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803720:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803726:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803732:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.803735:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803742:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803752:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:2.0:1787784899.803755:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784899.803758:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.803764:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc843c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.803863:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33497:x1874627658328704:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 17121us (17486us total) trans 4294968358 rc 0/0 00000100:00100000:2.0:1787784899.803870:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1580 00000100:00100000:0.0:1787784899.804608:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658329216 00000100:00100000:0.0:1787784899.804616:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658329216 00000100:00100000:0.0:1787784899.804687:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00100000:0.0:1787784899.804696:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658329216:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784899.804728:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 3/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804743:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.804748:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.804753:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804764:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804796:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804805:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804820:0:10688:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804826:0:10688:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784899.804830:0:10688:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804838:0:10688:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804848:0:10688:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.804856:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a7fead7/0xb0ce63137acc845f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.804902:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2357:x1874627658329216:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 207us (2366us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.804909:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00100000:1.0:1787784899.811192:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.811266:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658329728 00000100:00100000:0.0:1787784899.811275:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658329728 00000100:00100000:0.0:1787784899.812944:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 333 00000100:00100000:0.0:1787784899.812961:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+72:33497:x1874627658329728:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784899.813200:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+73:33497:x1874627658329728:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 245us (2016us total) trans 4294968359 rc 0/0 00000100:00100000:0.0:1787784899.813209:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 333 00000100:00100000:3.0:1787784899.813491:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.813914:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658329984 00000100:00100000:0.0:1787784899.813922:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658329984 00000100:00100000:0.0:1787784899.814041:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 334 00000100:00100000:0.0:1787784899.814049:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+73:33500:x1874627658329984:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1787784899.814423:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.814454:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.814595:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+74:33500:x1874627658329984:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 545us (1103us total) trans 4294968360 rc 0/0 00000100:00100000:1.0:1787784899.814604:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 334 00000100:00100000:2.0:1787784899.814818:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658330112 00000100:00100000:2.0:1787784899.814824:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658330112 00000100:00100000:2.0:1787784899.814852:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658330368 00000100:00100000:2.0:1787784899.814855:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658330368 00000100:00100000:2.0:1787784899.814879:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1583 00000100:00100000:2.0:1787784899.814886:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+75:33506:x1874627658330112:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784899.814898:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.814917:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc8466 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cc4 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.814974:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.814982:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1584 00010000:00010000:2.0:1787784899.814990:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.814994:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+76:33499:x1874627658330368:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784899.815009:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815023:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815030:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815038:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.815040:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815095:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815102:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815108:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815114:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.815118:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.815193:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 76 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.815202:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.815230:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 76 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.815243:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 76 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.815256:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668456576:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784899.815274:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815288:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.815292:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.815296:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815302:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc846d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815313:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815319:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.815406:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f3c0 going to sleep for 11 seconds 00010000:00010000:2.0:1787784899.815478:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815485:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.815488:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815495:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815504:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff989520131600) 00010000:00010000:2.0:1787784899.815506:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.815509:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815516:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8474 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815546:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc8466 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cc4 expref: 76 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815556:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.815558:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.815562:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc8466 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cc4 expref: 76 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.815570:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111600, rc 0) 00000100:00100000:2.0:1787784899.815597:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+75:33506:x1874627658330112:12345-192.168.203.43@tcp:101:cp.0 Request processed in 711us (1174us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784899.815603:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1583 00000100:00100000:3.0:1787784899.825834:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.825863:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668456576:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784899.825903:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:0.0:1787784899.825905:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.825997:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658330496 00000100:00100000:1.0:1787784899.826004:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658330496 00010000:00010000:1.0:1787784899.826036:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 75 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.826067:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 457 00000100:00100000:1.0:1787784899.826075:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+75:2354:x1874627658330496:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784899.826089:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784899.826094:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 75 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.826105:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 75 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.826130:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc8427 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cb6 expref: 75 pid: 5811 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.826180:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:1.0:1787784899.826186:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.826195:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.826205:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.826231:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784899.826269:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+74:2354:x1874627658330496:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 195us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.826691:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 457 00010000:00010000:1.0:1787784899.826715:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.826721:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.826927:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658330752 00000100:00100000:2.0:1787784899.826932:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658330752 00000100:00100000:2.0:1787784899.826957:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1585 00000100:00100000:2.0:1787784899.826963:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+75:33506:x1874627658330752:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784899.826987:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784899.827001:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc8482 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ccb expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.827096:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.827108:0:5811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 114 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.827117:0:5811:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.827143:0:5811:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 114 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.827174:0:5811:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 114 pid: 5813 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.827187:0:5811:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668456704:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784899.827573:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.827584:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.827590:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.827597:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.827602:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.827608:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.827620:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:1.0:1787784899.827623:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784899.827627:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.827634:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc847b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.827778:0:5811:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088fd80 going to sleep for 11 seconds 00000040:00080000:0.0:1787784899.827952:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784899.827957:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:1.0:1787784899.827988:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+76:33499:x1874627658330368:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 12994us (13529us total) trans 4294968361 rc 0/0 00000040:00080000:0.0:1787784899.827991:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 50 count 8 00000040:00080000:0.0:1787784899.827999:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784899.828004:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1584 00000100:00100000:1.0:1787784899.828624:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668456832:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784899.828663:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784899.842961:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.844617:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658331008 00000100:00100000:2.0:1787784899.844635:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658331008 00000100:00100000:2.0:1787784899.844721:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:2.0:1787784899.844733:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+77:33499:x1874627658331008:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784899.846569:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+78:33499:x1874627658331008:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 1826us (3609us total) trans 4294968362 rc 0/0 00000100:00100000:2.0:1787784899.846585:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 335 00000100:00100000:3.0:1787784899.856640:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668456832:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784899.857952:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.858249:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658331136 00000100:00100000:0.0:1787784899.858258:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658331136 00000100:00100000:2.0:1787784899.858548:0:5811:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5811:1874627668456704:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784899.858758:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 115 pid: 5813 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.859101:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 458 00000100:00100000:0.0:1787784899.859116:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:2357:x1874627658331136:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784899.859290:0:5811:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.859512:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784899.859522:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 115 pid: 5813 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.859536:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 115 pid: 5813 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.859565:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8378 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c62 expref: 115 pid: 5813 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.859584:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784899.859589:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.859607:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.859616:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.859664:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784899.859835:0:5811:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.859843:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.859873:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.859884:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.859892:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784899.861092:0:5811:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x22:0x0], index 1: 0 00000100:00100000:0.0:1787784899.861278:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:2357:x1874627658331136:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2160us (3329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.861291:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 458 00010000:00010000:2.0:1787784899.863523:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863555:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863571:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863633:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863640:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863648:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863654:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784899.863659:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863671:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863686:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784899.863689:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784899.863695:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.863701:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8489 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784899.863720:0:5811:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008c9e2032 reply state 000000003de69cba transno 4294968363 ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.863732:0:5811:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784899.864979:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc8482 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 78 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.865018:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 77 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.865028:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784899.865031:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784899.865035:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111e00, rc 0) 00000100:00100000:2.0:1787784899.865749:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+78:33506:x1874627658330752:12345-192.168.203.43@tcp:101:cp.0 Request processed in 38777us (39917us total) trans 4294968363 rc 0/0 00000100:00100000:2.0:1787784899.865763:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1585 00000100:00100000:1.0:1787784899.875044:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.875220:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:1.0:1787784899.889055:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889078:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889087:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889094:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784899.889100:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889108:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889133:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff989520131600) 00010000:00010000:1.0:1787784899.889163:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784899.889170:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889177:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8490 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.889359:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.889701:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658331520 00000100:00100000:1.0:1787784899.889715:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658331520 00000100:00100000:1.0:1787784899.889796:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 470 00000100:00100000:1.0:1787784899.889809:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+14:2354:x1874627658331520:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784899.889833:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.889858:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cd2 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784899.889891:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x22:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784899.889916:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1787784899.889921:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784899.889932:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 15 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889950:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 15 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.889964:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfe00, rc 0) 00000100:00100000:1.0:1787784899.890034:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2354:x1874627658331520:12345-192.168.203.43@tcp:101:cp.0 Request processed in 227us (681us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.890042:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 470 00000100:00100000:3.0:1787784899.896642:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.897480:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658331264 00000100:00100000:2.0:1787784899.897497:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658331264 00000100:00100000:2.0:1787784899.897579:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658331392 00000100:00100000:2.0:1787784899.897583:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658331392 00000100:00100000:2.0:1787784899.897613:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1586 00000100:00100000:2.0:1787784899.897625:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+78:33508:x1874627658331264:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784899.897675:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784899.897830:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+78:33508:x1874627658331264:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 207us (22800us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784899.897839:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1586 00000100:00100000:2.0:1787784899.897862:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1587 00000100:00100000:2.0:1787784899.897870:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+77:33507:x1874627658331392:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784899.897883:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x31:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784899.897926:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+77:33507:x1874627658331392:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 57us (22707us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784899.897933:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1587 00000100:00100000:1.0:1787784899.914906:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658331648 00000100:00100000:1.0:1787784899.914926:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658331648 00000100:00100000:0.0:1787784899.915184:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 471 00000100:00100000:0.0:1787784899.915201:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+15:33506:x1874627658331648:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784899.915226:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784899.915253:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 2/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cd9 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.915296:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 16 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.915320:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784899.915326:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.915333:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 16 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.915389:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 16 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.915411:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 16 pid: 6671 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.915437:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668456960:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784899.915563:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784899.923847:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668456960:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784899.923909:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 3/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 16 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.923934:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f800, rc 0) 00000100:00100000:0.0:1787784899.924012:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+16:33506:x1874627658331648:12345-192.168.203.43@tcp:101:cp.0 Request processed in 8815us (27384us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.924023:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 471 00000100:00100000:2.0:1787784899.939014:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.940869:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658331776 00000100:00100000:0.0:1787784899.940890:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658331776 00010000:00010000:0.0:1787784899.940961:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 16 pid: 6671 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.941035:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:0.0:1787784899.941048:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2356:x1874627658331776:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784899.941071:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784899.941105:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 16 pid: 6671 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.941130:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 16 pid: 6671 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.941228:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dadf0795/0xb0ce63137acc84a5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd2 expref: 16 pid: 6671 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.941253:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.941258:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784899.941268:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 15 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.941279:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 15 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.941330:0:11303:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 15 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.941346:0:11303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11303:1874627668457088:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784899.941713:0:11303:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f240 going to sleep for 11 seconds 00000100:00100000:2.0:1787784899.945359:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.945481:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658332032 00000100:00100000:1.0:1787784899.945495:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658332032 00000100:00100000:1.0:1787784899.945579:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1588 00000100:00100000:1.0:1787784899.945592:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+77:33507:x1874627658332032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784899.945626:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784899.945654:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc84b3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ce0 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.945804:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc84ba lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.945822:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc84ba lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.945840:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc84ba lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784899.945854:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x31:0x0]: returning size 158 00010000:00010000:1.0:1787784899.946016:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc84b3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ce0 expref: 80 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.946040:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc84ba lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ce0 expref: 79 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.946050:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:1.0:1787784899.946054:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784899.946058:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df000, rc 0) 00000100:00100000:1.0:1787784899.948003:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+79:33507:x1874627658332032:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 558us (799us total) trans 4294968364 rc 0/0 00000100:00100000:1.0:1787784899.948022:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1588 00000100:00100000:2.0:1787784899.956037:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784899.956136:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.956409:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658332544 00000100:00100000:1.0:1787784899.956422:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658332544 00000100:00100000:3.0:1787784899.956442:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658332288 00000100:00100000:3.0:1787784899.956448:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658332288 00000100:00100000:3.0:1787784899.956496:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1589 00000100:00100000:1.0:1787784899.956497:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 336 00000100:00100000:1.0:1787784899.956513:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33506:x1874627658332544:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:3.0:1787784899.956515:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33508:x1874627658332288:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784899.956543:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784899.956573:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc84c8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ce7 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.956686:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc84cf lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.956711:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc84cf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.956726:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc84cf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784899.956739:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x32:0x0]: returning size 158 00010000:00010000:3.0:1787784899.956870:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc84c8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ce7 expref: 84 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.956894:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc84cf lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ce7 expref: 83 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.956904:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:3.0:1787784899.956907:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784899.956912:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:1.0:1787784899.957060:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+83:33506:x1874627658332544:12345-192.168.203.43@tcp:35:cp.0 Request processed in 550us (922us total) trans 4294968365 rc 0/0 00000100:00100000:1.0:1787784899.957069:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 336 00000100:00100000:3.0:1787784899.957234:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+82:33508:x1874627658332288:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 480us (967us total) trans 4294968366 rc 0/0 00000100:00100000:3.0:1787784899.957244:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1589 00000100:00100000:0.0:1787784899.958020:0:11303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11303:1874627668457088:192.168.203.43@tcp:105: 00010000:00010000:0.0:1787784899.958067:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784899.958139:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2356:x1874627658331776:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 17087us (19134us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.958187:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 459 00000100:00100000:3.0:1787784899.960480:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.960541:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.960973:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658332800 00000100:00100000:1.0:1787784899.960996:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658332800 00000100:00100000:1.0:1787784899.961082:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 71 00000100:00100000:1.0:1787784899.961093:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c97450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+82:2357:x1874627658332800:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784899.961125:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784899.961205:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc84ba lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ce0 expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.961228:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:1.0:1787784899.961233:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784899.961297:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c97450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+81:2357:x1874627658332800:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 206us (755us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.961305:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 71 00000100:00100000:2.0:1787784899.961679:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658332672 00000100:00100000:2.0:1787784899.961685:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658332672 00000100:00100000:2.0:1787784899.961735:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 472 00000100:00100000:2.0:1787784899.961744:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2355:x1874627658332672:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784899.961943:0:22706:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.961974:0:22706:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 15 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.961984:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x21:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784899.961989:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784899.961995:0:22706:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 4/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.962040:0:22706:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 15 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.962057:0:22706:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 15 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.962075:0:22706:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668457216:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784899.962293:0:22706:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1a9f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787784899.969544:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.969595:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.969733:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658333056 00000100:00100000:3.0:1787784899.969746:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658333056 00000100:00100000:3.0:1787784899.969834:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 72 00000100:00100000:3.0:1787784899.969845:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+81:2355:x1874627658333056:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784899.969865:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784899.969898:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc84cf lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ce7 expref: 81 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784899.969922:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:3.0:1787784899.969928:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784899.969997:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+80:2355:x1874627658333056:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 154us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784899.970005:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 72 00000100:00100000:2.0:1787784899.970506:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658332928 00000100:00100000:2.0:1787784899.970513:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658332928 00000100:00100000:2.0:1787784899.970557:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 473 00000100:00100000:2.0:1787784899.970565:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658332928:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784899.970609:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.970637:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 15 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.970654:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1f:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784899.970659:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784899.970664:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.970705:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 15 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.970721:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 15 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784899.970739:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668457344:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784899.970834:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784899.970891:0:22706:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668457216:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784899.970916:0:22706:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784899.984718:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784899.984811:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784899.984890:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668457344:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784899.984925:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.984985:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658333184 00000100:00100000:0.0:1787784899.985007:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658333184 00002000:00100000:0.0:1787784899.985067:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x21:0x0] (0->157) req@ffff98954073ad00 x1874627658333184/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.985079:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954073ad00 with x1874627658333184 ext(0->157) 00010000:00010000:0.0:1787784899.985089:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 16 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.985105:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 16 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.985115:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff98954073ad00 00000100:00100000:2.0:1787784899.985117:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658333312 00000100:00100000:2.0:1787784899.985124:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658333312 00000100:00100000:0.0:1787784899.985185:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 20 00002000:00100000:2.0:1787784899.985190:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1f:0x0] (0->157) req@ffff989540739a40 x1874627658333312/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/0 e 0 to 0 dl 1787784910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1787784899.985202:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2357:x1874627658333184:12345-192.168.203.43@tcp:4:cp.0 00002000:00100000:2.0:1787784899.985205:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540739a40 with x1874627658333312 ext(0->157) 00010000:00010000:2.0:1787784899.985214:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 16 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.985232:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 16 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784899.985239:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989540739a40 00000100:00100000:2.0:1787784899.985256:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 21 00000100:00100000:2.0:1787784899.985265:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2357:x1874627658333312:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784899.985986:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2357:x1874627658333312:12345-192.168.203.43@tcp:4:cp.0 Request processed in 719us (1171us total) trans 4294967378 rc 0/0 00000100:00100000:2.0:1787784899.985996:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 21 00002000:00100000:2.0:1787784899.986005:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1f:0x0] (0->157) req@ffff989540739a40 x1874627658333312/t4294967378(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/448 e 0 to 0 dl 1787784910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784899.986014:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989540739a40 with x1874627658333312 ext(0->157) 00010000:00010000:2.0:1787784899.986021:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 18 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784899.986031:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 18 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784899.986037:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989540739a40 00000100:00100000:0.0:1787784899.986126:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2357:x1874627658333184:12345-192.168.203.43@tcp:4:cp.0 Request processed in 928us (1431us total) trans 4294967392 rc 0/0 00000100:00100000:0.0:1787784899.986284:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 20 00002000:00100000:0.0:1787784899.986292:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x21:0x0] (0->157) req@ffff98954073ad00 x1874627658333184/t4294967392(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:25/0 lens 648/448 e 0 to 0 dl 1787784910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787784899.986300:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954073ad00 with x1874627658333184 ext(0->157) 00010000:00010000:0.0:1787784899.986308:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 18 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.986320:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 18 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784899.986333:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff98954073ad00 00000100:00100000:3.0:1787784899.994120:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784899.995551:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658333440 00000100:00100000:0.0:1787784899.995568:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658333440 00010000:00010000:0.0:1787784899.995630:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 18 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784899.995686:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:0.0:1787784899.995710:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2354:x1874627658333440:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784899.995729:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784899.995758:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 18 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.995774:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 18 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.996275:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc82de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c00 expref: 18 pid: 13825 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.996297:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.996301:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784899.996308:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.996320:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784899.996384:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784899.996508:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996519:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996550:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996560:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996567:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996575:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784899.996579:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996588:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996604:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784899.996611:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc84e4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784899.996720:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658332928:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 26154us (27190us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784899.996730:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 473 00000100:00100000:0.0:1787784899.997051:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2354:x1874627658333440:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1354us (2945us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784899.997062:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 460 00000100:00100000:1.0:1787784900.001894:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.002034:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658333568 00000100:00100000:0.0:1787784900.002049:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658333568 00010000:00010000:0.0:1787784900.002128:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 18 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.002224:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 461 00000100:00100000:0.0:1787784900.002236:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2357:x1874627658333568:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784900.002256:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784900.002286:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 18 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.002303:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 18 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.002333:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e54dea0/0xb0ce63137acc81e2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b6d expref: 18 pid: 10688 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.002353:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784900.002359:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784900.002365:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.002383:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.002444:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784900.002510:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658333568:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 277us (631us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784900.002519:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 461 00010000:00010000:1.0:1787784900.002805:0:22706:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002817:0:22706:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002848:0:22706:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002856:0:22706:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002864:0:22706:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002878:0:22706:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110a00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784900.002882:0:22706:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002891:0:22706:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002904:0:22706:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.002912:0:22706:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc84dd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784900.002965:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2355:x1874627658332672:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 41222us (42499us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.002974:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 472 00000100:00080000:2.0:1787784900.003196:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784900.006246:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.006359:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658333824 00000100:00100000:0.0:1787784900.006372:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658333824 00000100:00100000:0.0:1787784900.006464:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1590 00000100:00100000:0.0:1787784900.006477:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33508:x1874627658333824:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784900.006628:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407383c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33508:x1874627658333824:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 153us (395us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784900.006637:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1590 00000100:00080000:2.0:1787784900.006776:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784900.015293:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.016104:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658334080 00000100:00100000:1.0:1787784900.016121:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658334080 00000100:00100000:1.0:1787784900.016240:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1591 00000100:00100000:1.0:1787784900.016251:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33507:x1874627658334080:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784900.016398:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33507:x1874627658334080:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 148us (1122us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.016407:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1591 00000100:00100000:3.0:1787784900.017968:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.023025:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658334208 00000100:00100000:1.0:1787784900.023046:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658334208 00000100:00100000:1.0:1787784900.023402:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 474 00000100:00100000:1.0:1787784900.023430:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2356:x1874627658334208:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784900.023453:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.023480:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cee expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784900.023514:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1f:0x0] initial LVB size: 158, mtime: 0x6a8f6ec2, atime: 0x0, ctime: 0x6a8f6ec2, blocks: 0x1 00010000:00010000:1.0:1787784900.023542:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1787784900.023548:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784900.023559:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.023581:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.023598:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de000, rc 0) 00000100:00100000:1.0:1787784900.023682:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658334208:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 270us (5723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.023690:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 474 00000100:00080000:2.0:1787784900.024373:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784900.030298:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.031022:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658334336 00000100:00100000:1.0:1787784900.031035:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658334336 00000100:00100000:1.0:1787784900.031112:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 475 00000100:00100000:1.0:1787784900.031125:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2355:x1874627658334336:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784900.031198:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.031228:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cf5 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784900.031264:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x21:0x0] initial LVB size: 158, mtime: 0x6a8f6ec2, atime: 0x0, ctime: 0x6a8f6ec2, blocks: 0x1 00010000:00010000:1.0:1787784900.031290:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1787784900.031296:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784900.031307:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.031326:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.031342:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfa00, rc 0) 00000100:00100000:1.0:1787784900.031429:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2355:x1874627658334336:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 306us (1145us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.031437:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 475 00000100:00100000:3.0:1787784900.035606:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.042295:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.051756:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.053605:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658334592 00000100:00100000:2.0:1787784900.053623:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658334592 00000100:00100000:2.0:1787784900.053721:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658334848 00000100:00100000:2.0:1787784900.053726:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658334848 00000100:00100000:2.0:1787784900.053754:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1592 00000100:00100000:2.0:1787784900.053765:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+81:33508:x1874627658334592:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784900.053926:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+81:33508:x1874627658334592:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 161us (18334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784900.053934:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1592 00000100:00100000:2.0:1787784900.053956:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1593 00000100:00100000:2.0:1787784900.053962:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33507:x1874627658334848:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784900.054017:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+80:33507:x1874627658334848:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 54us (11736us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784900.054023:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1593 00000100:00100000:0.0:1787784900.063521:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.063888:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658335104 00000100:00100000:2.0:1787784900.063902:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658335104 00002000:00100000:2.0:1787784900.063965:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x21:0x0] (0->4095) req@ffff989401cc3840 x1874627658335104/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:26/0 lens 488/0 e 0 to 0 dl 1787784911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784900.063978:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cc3840 with x1874627658335104 ext(0->4095) 00002000:00010000:2.0:1787784900.063984:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401cc3840 00000100:00100000:2.0:1787784900.064041:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 128 00000100:00100000:2.0:1787784900.064052:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658335104:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784900.065272:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658335104:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 1210us (1756us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784900.065287:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 128 00002000:00100000:2.0:1787784900.065299:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x21:0x0] (0->4095) req@ffff989401cc3840 x1874627658335104/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:26/0 lens 488/4536 e 0 to 0 dl 1787784911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784900.065310:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989401cc3840 with x1874627658335104 ext(0->4095) 00002000:00010000:2.0:1787784900.065316:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989401cc3840 00000100:00100000:2.0:1787784900.066298:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.066418:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658335232 00000100:00100000:3.0:1787784900.066429:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658335232 00002000:00100000:3.0:1787784900.066481:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1f:0x0] (0->4095) req@ffff98954028d680 x1874627658335232/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:26/0 lens 488/0 e 0 to 0 dl 1787784911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784900.066495:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954028d680 with x1874627658335232 ext(0->4095) 00002000:00010000:3.0:1787784900.066500:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98954028d680 00000100:00100000:3.0:1787784900.066539:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 129 00000100:00100000:3.0:1787784900.066550:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658335232:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1787784900.066713:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658334976 00000100:00100000:1.0:1787784900.066720:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658334976 00000100:00080000:0.0:1787784900.066879:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784900.066963:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658335232:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 413us (673us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784900.066973:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 129 00002000:00100000:3.0:1787784900.066981:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1f:0x0] (0->4095) req@ffff98954028d680 x1874627658335232/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:26/0 lens 488/4536 e 0 to 0 dl 1787784911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784900.066991:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98954028d680 with x1874627658335232 ext(0->4095) 00002000:00010000:3.0:1787784900.066994:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff98954028d680 00000100:00100000:1.0:1787784900.067244:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1594 00000100:00100000:1.0:1787784900.067256:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33511:x1874627658334976:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784900.067271:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.067295:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc84f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1cfc expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067386:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067404:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067533:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067552:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067560:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067567:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067673:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc850e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067681:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc850e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067688:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc850e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784900.067736:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 158 00000004:00200000:1.0:1787784900.067741:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784900.067757:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc850e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067770:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc850e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067779:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067899:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067906:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067914:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.067919:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067928:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067942:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.067946:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.067951:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067959:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8500 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067973:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.067980:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068249:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068257:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.068261:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068269:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068278:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff98951b73bf00) 00010000:00010000:1.0:1787784900.068281:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.068284:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068290:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8507 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068332:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc84f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cfc expref: 116 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068348:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc850e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cfc expref: 115 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.068356:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784900.068358:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784900.068361:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de400, rc 0) 00000100:00100000:1.0:1787784900.068611:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33511:x1874627658334976:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1357us (16868us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.068620:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1594 00000100:00100000:0.0:1787784900.077637:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.077758:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658335488 00000100:00100000:1.0:1787784900.077772:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658335488 00000100:00100000:1.0:1787784900.077878:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 476 00000100:00100000:1.0:1787784900.077890:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658335488:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784900.077915:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.077941:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000964ca472/0xb0ce63137acc8515 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d0a expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784900.078029:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668457472:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784900.078205:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4240 going to sleep for 11 seconds 00000100:00080000:0.0:1787784900.079347:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784900.080835:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.080887:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.081242:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658335360 00000100:00100000:1.0:1787784900.081249:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658335616 00000100:00100000:3.0:1787784900.081254:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658335360 00000100:00100000:1.0:1787784900.081257:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658335616 00000100:00100000:2.0:1787784900.081508:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668457600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784900.081575:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784900.081637:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668457600 00000100:00100000:2.0:1787784900.081640:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668457600 00000100:00100000:2.0:1787784900.081685:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1787784900.081707:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668457600:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784900.081735:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 103743488 pending 5111808 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784900.081803:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668457600:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (227us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784900.081811:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1787784900.081879:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668457600:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784900.082096:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1595 00000100:00100000:1.0:1787784900.082107:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+81:33508:x1874627658335616:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784900.082126:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.082190:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc851c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d11 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082234:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082251:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082264:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082395:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc851c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082416:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082432:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:1.0:1787784900.082435:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784900.082440:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df800, rc 0) 00000100:00100000:1.0:1787784900.082505:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+82:33508:x1874627658335616:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 400us (1617us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.082512:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1595 00000100:00100000:1.0:1787784900.082531:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1596 00000100:00100000:1.0:1787784900.082539:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+81:33507:x1874627658335360:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784900.082547:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.082557:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc852a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d03 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082573:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082583:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082590:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082647:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc852a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082660:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 82 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.082674:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:1.0:1787784900.082676:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1787784900.082680:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df000, rc 0) 00000100:00100000:1.0:1787784900.082715:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+82:33507:x1874627658335360:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 177us (1892us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.082722:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1596 00000100:00080000:0.0:1787784900.092972:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1787784900.094907:0:6671:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1787784900.094920:0:6671:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb mtime: 0 -> 1787784899 00002000:00010000:0.0:1787784900.094924:0:6671:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb atime: 0 -> 1787784899 00002000:00010000:0.0:1787784900.094926:0:6671:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb ctime: 0 -> 1787784899 00000100:00100000:0.0:1787784900.094959:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668457472:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784900.095023:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000964ca472/0xb0ce63137acc8515 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d0a expref: 37 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.095057:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000964ca472/0xb0ce63137acc8515 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d0a expref: 37 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.095076:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dea00, rc 0) 00000100:00100000:0.0:1787784900.095718:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658335488:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 17826us (18089us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784900.095730:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 476 00000100:00100000:2.0:1787784900.097716:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.097934:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658335872 00000100:00100000:3.0:1787784900.097947:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658335872 00000100:00100000:3.0:1787784900.098025:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1597 00000100:00100000:3.0:1787784900.098037:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+82:33508:x1874627658335872:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784900.098078:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784900.098402:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+82:33508:x1874627658335872:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 365us (704us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.098611:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1597 00000100:00100000:2.0:1787784900.098722:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.099335:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658335744 00000100:00100000:3.0:1787784900.099342:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658335744 00000100:00100000:3.0:1787784900.099386:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1598 00000100:00100000:3.0:1787784900.099396:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+82:33507:x1874627658335744:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784900.099524:0:5812:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x31:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784900.099611:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+82:33507:x1874627658335744:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 216us (888us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.099619:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1598 00000100:00100000:2.0:1787784900.111875:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.111982:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.112622:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658336000 00000100:00100000:1.0:1787784900.112636:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658336000 00000100:00100000:1.0:1787784900.112966:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658336128 00000100:00100000:1.0:1787784900.112971:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658336128 00000100:00100000:3.0:1787784900.113409:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.113456:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658336256 00000100:00100000:3.0:1787784900.113461:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658336256 00000100:00100000:3.0:1787784900.113503:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1599 00000100:00100000:3.0:1787784900.113515:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33511:x1874627658336256:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784900.113542:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.113570:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8538 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d18 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.113658:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.113689:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.113714:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784900.113808:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:3.0:1787784900.113813:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 6 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784900.113841:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.113874:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8538 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 117 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.113898:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 116 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.113909:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.113912:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784900.113917:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784900.114529:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540739680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33511:x1874627658336256:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1012us (1120us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.114539:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1599 00000100:00100000:0.0:1787784900.115034:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 477 00000100:00100000:0.0:1787784900.115051:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658336000:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784900.115091:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115108:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784900.115113:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784900.115120:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115162:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115200:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115210:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115219:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115227:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784900.115232:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115242:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115259:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115268:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002038a015/0xb0ce63137acc8546 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.115337:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658336000:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 291us (3475us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784900.115345:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 477 00000100:00100000:0.0:1787784900.115375:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 478 00000100:00100000:0.0:1787784900.115385:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658336128:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784900.115410:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 3/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115431:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784900.115435:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784900.115439:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115449:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115469:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115477:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115484:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115490:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784900.115493:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115506:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115516:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.115524:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009d7e20e8/0xb0ce63137acc854d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.115556:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2357:x1874627658336128:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 173us (3573us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784900.115563:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 478 00000100:00100000:3.0:1787784900.122916:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.124103:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.124183:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658336512 00000100:00100000:3.0:1787784900.124196:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658336512 00000100:00100000:3.0:1787784900.124261:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658336768 00000100:00100000:3.0:1787784900.124264:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658336768 00000100:00100000:3.0:1787784900.124288:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1600 00000100:00100000:3.0:1787784900.124299:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+83:33508:x1874627658336512:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784900.124619:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.124642:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.124652:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.124700:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.124717:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 83 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.124736:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668457728:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784900.124852:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787784900.124878:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1601 00000100:00100000:3.0:1787784900.124887:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+83:33507:x1874627658336768:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784900.124926:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.124939:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.124946:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.124977:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 83 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.125036:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 83 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.125048:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ed2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668457856:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784900.125080:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784900.129709:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668457728:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784900.129755:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784900.138926:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.138949:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.138984:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.139020:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658337152 00000100:00100000:2.0:1787784900.139031:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658337152 00000100:00100000:2.0:1787784900.139098:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 479 00000100:00100000:2.0:1787784900.139109:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2354:x1874627658337152:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784900.139172:0:22706:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784900.139200:0:22706:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc8562 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d1f expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784900.139279:0:22706:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668457984:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784900.139397:0:22706:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787784900.139814:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658336896 00000100:00100000:3.0:1787784900.139820:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658336896 00010000:00010000:3.0:1787784900.139855:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 84 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.139891:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658337024 00000100:00100000:3.0:1787784900.139894:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658337024 00010000:00010000:3.0:1787784900.139909:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 85 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.139932:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 462 00000100:00100000:3.0:1787784900.139941:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+85:2357:x1874627658336896:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784900.139953:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784900.139960:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 85 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.139976:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 85 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140001:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8523 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d11 expref: 85 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140023:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:3.0:1787784900.140028:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140038:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140047:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140075:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784900.140113:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+84:2357:x1874627658336896:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 173us (1202us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.140121:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 462 00000100:00100000:3.0:1787784900.140169:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 463 00000100:00100000:3.0:1787784900.140178:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+83:2357:x1874627658337024:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784900.140187:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784900.140193:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 83 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140203:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 83 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140215:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:3.0:1787784900.140219:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140227:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140233:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140244:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784900.140264:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+83:2357:x1874627658337024:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 87us (1313us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.140270:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 463 00010000:00010000:3.0:1787784900.140303:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140311:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140578:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140587:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140594:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140601:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.140606:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140614:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140624:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:3.0:1787784900.140627:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784900.140630:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.140636:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8554 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.140796:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+83:33508:x1874627658336512:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 16498us (17895us total) trans 4294968367 rc 0/0 00000100:00100000:3.0:1787784900.140808:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1600 00000100:00100000:1.0:1787784900.148238:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.148555:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658337408 00000100:00100000:3.0:1787784900.148570:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658337408 00000100:00100000:3.0:1787784900.148634:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:3.0:1787784900.148645:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+83:33508:x1874627658337408:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00010000:00010000:0.0:1787784900.150439:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8531 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d03 expref: 83 pid: 5811 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.150471:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ed2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668457856:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784900.150504:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.150517:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.150524:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.151330:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.151343:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.151351:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.151359:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.151364:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.151373:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.151395:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:0.0:1787784900.151399:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784900.151404:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.151513:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc855b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784900.151937:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33508:x1874627658337408:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 3288us (3707us total) trans 4294968369 rc 0/0 00000100:00100000:2.0:1787784900.151951:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 337 00000100:00100000:0.0:1787784900.152269:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+83:33507:x1874627658336768:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 27382us (28171us total) trans 4294968368 rc 0/0 00000100:00100000:0.0:1787784900.152278:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1601 00000100:00100000:2.0:1787784900.152755:0:22706:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668457984:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784900.152900:0:22706:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc8562 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d1f expref: 37 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.152927:0:22706:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc8562 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d1f expref: 37 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.152942:0:22706:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110a00, rc 0) 00000100:00100000:2.0:1787784900.153025:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2354:x1874627658337152:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 13917us (14039us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784900.153034:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 479 00000040:00080000:2.0:1787784900.155583:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784900.155595:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784900.156121:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 8 00000040:00080000:2.0:1787784900.157977:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784900.160043:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.160528:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658337664 00000100:00100000:0.0:1787784900.160542:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658337664 00000100:00100000:0.0:1787784900.160613:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 338 00000100:00100000:0.0:1787784900.160624:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+83:33507:x1874627658337664:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1787784900.160976:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33507:x1874627658337664:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 352us (946us total) trans 4294968370 rc 0/0 00000100:00100000:0.0:1787784900.160986:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 338 00000100:00100000:2.0:1787784900.174736:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.175121:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658337920 00000100:00100000:3.0:1787784900.175178:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658337920 00000100:00100000:3.0:1787784900.175248:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1602 00000100:00100000:3.0:1787784900.175260:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33511:x1874627658337920:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:3.0:1787784900.175580:0:20438:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].50, 136 off14992 00000100:00100000:3.0:1787784900.177102:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33511:x1874627658337920:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 1830us (2365us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.177118:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1602 00000100:00100000:2.0:1787784900.205726:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.213388:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658338048 00000100:00100000:1.0:1787784900.213410:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658338048 00000100:00100000:1.0:1787784900.213714:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1603 00000100:00100000:1.0:1787784900.213729:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33512:x1874627658338048:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784900.213760:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.213812:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8569 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d26 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.213901:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.213919:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.213932:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.213954:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.213960:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.213967:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214068:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214077:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214083:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214089:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.214094:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214102:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214117:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.214121:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.214126:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214530:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8570 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214553:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214559:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214565:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214570:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.214573:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214579:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214590:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff98951b73bf00) 00010000:00010000:1.0:1787784900.214592:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.214595:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214601:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc8577 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214649:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8569 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d26 expref: 117 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214661:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.214663:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.214668:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8569 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d26 expref: 117 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.214679:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df600, rc 0) 00000100:00100000:1.0:1787784900.214885:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33512:x1874627658338048:12345-192.168.203.43@tcp:101:bash.0 Request processed in 1158us (9173us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784900.214894:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1603 00000100:00100000:0.0:1787784900.246043:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.246205:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658338176 00000100:00100000:3.0:1787784900.246217:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658338176 00000100:00100000:3.0:1787784900.246294:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1604 00000100:00100000:3.0:1787784900.246305:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33513:x1874627658338176:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784900.246338:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.246363:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc857e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d2d expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246478:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246496:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246514:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246532:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246539:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246545:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246632:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246641:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246647:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246653:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.246658:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246666:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246680:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.246683:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.246688:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246696:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8585 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246712:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246717:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246723:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246728:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.246731:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246737:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246744:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff9895198b5100) 00010000:00010000:3.0:1787784900.246746:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.246749:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246754:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc858c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246805:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc857e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d2d expref: 85 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246822:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.246824:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.246828:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc857e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d2d expref: 85 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.246838:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784900.246918:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33513:x1874627658338176:12345-192.168.203.43@tcp:101:rm.0 Request processed in 614us (890us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784900.246926:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1604 00000100:00100000:2.0:1787784900.254329:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.254494:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658338304 00000100:00100000:3.0:1787784900.254506:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658338304 00000100:00100000:3.0:1787784900.254595:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1605 00000100:00100000:3.0:1787784900.254609:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33513:x1874627658338304:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784900.254639:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.254668:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8593 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d34 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254790:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254810:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254824:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254842:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254851:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254858:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254946:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254955:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254962:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254969:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.254974:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.254983:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255330:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.255334:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.255340:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255348:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc859a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255366:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255373:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255379:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255385:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.255389:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255396:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255404:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff9895198b5600) 00010000:00010000:3.0:1787784900.255407:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.255410:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255416:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255462:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8593 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d34 expref: 85 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255475:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.255478:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.255482:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8593 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d34 expref: 85 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.255499:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784900.255583:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088503c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33513:x1874627658338304:12345-192.168.203.43@tcp:101:rm.0 Request processed in 977us (1271us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784900.255592:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1605 00000100:00100000:0.0:1787784900.284843:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.285008:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658338432 00000100:00100000:1.0:1787784900.285022:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658338432 00000100:00100000:1.0:1787784900.285098:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1606 00000100:00100000:1.0:1787784900.285110:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33514:x1874627658338432:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784900.285142:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.285220:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc85a8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d3b expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285308:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285325:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285338:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285354:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285371:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285379:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285463:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285471:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285479:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285485:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.285490:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285499:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285512:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.285515:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.285521:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285528:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc85af lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285543:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285550:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285556:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285563:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.285566:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285572:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285578:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff98951b73bf00) 00010000:00010000:1.0:1787784900.285614:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.285618:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285625:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc85b6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285665:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc85a8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d3b expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285677:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.285679:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.285683:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de800/0xb0ce63137acc85a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d3b expref: 85 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.285694:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de800, rc 0) 00000100:00100000:1.0:1787784900.285767:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33514:x1874627658338432:12345-192.168.203.43@tcp:101:cp.0 Request processed in 658us (940us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784900.285974:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1606 00000100:00100000:1.0:1787784900.290850:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.291009:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658338560 00000100:00100000:2.0:1787784900.291022:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658338560 00000100:00100000:2.0:1787784900.291094:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1607 00000100:00100000:2.0:1787784900.291104:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33514:x1874627658338560:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784900.291135:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784900.291209:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc85bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d42 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291300:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291318:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291333:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291351:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291358:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291364:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291459:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291473:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291480:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291487:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784900.291492:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291499:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291514:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784900.291518:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784900.291523:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291529:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc85c4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291543:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291548:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291554:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291558:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784900.291562:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291567:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291572:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff989520131600) 00010000:00010000:2.0:1787784900.291575:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784900.291578:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291583:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc85cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291617:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc85bd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d42 expref: 85 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291627:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784900.291629:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784900.291632:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc85bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d42 expref: 85 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.291642:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111600, rc 0) 00000100:00100000:2.0:1787784900.291714:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33514:x1874627658338560:12345-192.168.203.43@tcp:101:cp.0 Request processed in 611us (879us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784900.291723:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1607 00000100:00100000:0.0:1787784900.300834:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.300947:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658338816 00000100:00100000:3.0:1787784900.300961:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658338816 00000100:00100000:3.0:1787784900.301038:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1608 00000100:00100000:3.0:1787784900.301051:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+84:33514:x1874627658338816:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784900.301080:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.301106:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc85d2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d49 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.301331:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.301349:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 115 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.301359:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.301404:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 115 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.301421:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 115 pid: 10850 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.301443:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668458112:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784900.301840:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1696c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784900.311058:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668458112:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784900.311102:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.312394:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.312708:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658338944 00000100:00100000:3.0:1787784900.312720:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658338944 00010000:00010000:3.0:1787784900.312774:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 116 pid: 10850 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.313078:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 464 00000100:00100000:3.0:1787784900.313091:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:2357:x1874627658338944:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784900.313110:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784900.313119:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 116 pid: 10850 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.313188:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 116 pid: 10850 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.313217:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc853f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d18 expref: 116 pid: 10850 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.313238:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.313244:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.313256:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.313265:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.313291:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784900.313374:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:2357:x1874627658338944:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 278us (978us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.313382:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 464 00010000:00010000:0.0:1787784900.314977:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.314994:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.315033:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.315049:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.315061:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784900.315761:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x20:0x0], index 0: 0 00010000:00010000:0.0:1787784900.319580:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.319609:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.319624:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.320573:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.320588:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.320597:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.320603:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.320609:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.320617:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.320635:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.320638:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.320644:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.321948:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc85d9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784900.321990:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000074c9a50c reply state 00000000163e674f transno 4294968371 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.322003:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784900.322078:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc85d2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 87 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.322098:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 86 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.322113:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:0.0:1787784900.322116:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784900.322120:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f200, rc 0) 00000100:00100000:0.0:1787784900.323091:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33514:x1874627658338816:12345-192.168.203.43@tcp:101:cp.0 Request processed in 22034us (22263us total) trans 4294968371 rc 0/0 00000100:00100000:0.0:1787784900.323103:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1608 00010000:00010000:1.0:1787784900.328360:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.328395:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.328405:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.328412:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.328418:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.328427:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.328448:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff989520108100) 00010000:00010000:1.0:1787784900.328452:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.328458:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.328465:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc85e0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784900.328919:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.329335:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658339072 00000100:00100000:1.0:1787784900.329348:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658339072 00000100:00100000:1.0:1787784900.329433:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 480 00000100:00100000:1.0:1787784900.329446:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658339072:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784900.329470:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.329497:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d50 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784900.329537:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x20:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1787784900.329562:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1787784900.329567:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784900.329579:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 18 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.329597:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 18 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.329611:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df400, rc 0) 00000100:00100000:1.0:1787784900.329692:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2356:x1874627658339072:12345-192.168.203.43@tcp:101:cp.0 Request processed in 249us (779us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.329700:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 480 00000100:00100000:0.0:1787784900.341513:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.341702:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658339200 00000100:00100000:3.0:1787784900.341717:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658339200 00000100:00100000:0.0:1787784900.342026:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 481 00000100:00100000:0.0:1787784900.342040:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+18:33514:x1874627658339200:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784900.342066:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784900.342091:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d57 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.342209:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 19 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.342221:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784900.342227:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784900.342233:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.342278:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 19 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.344043:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 19 pid: 10688 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.344076:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668458240:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784900.344337:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784900.347595:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.347964:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658339328 00000100:00100000:3.0:1787784900.347974:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658339328 00010000:00010000:3.0:1787784900.348022:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 20 pid: 10688 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.348129:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 465 00000100:00100000:3.0:1787784900.348190:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c94050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658339328:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784900.348207:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784900.348235:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 20 pid: 10688 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.348250:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 20 pid: 10688 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.348277:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000321934e4/0xb0ce63137acc85f5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d50 expref: 20 pid: 10688 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.348297:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784900.348301:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784900.348310:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.348322:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.348366:0:11303:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.348392:0:11303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668458368:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784900.348483:0:11303:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1696c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784900.349015:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668458240:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784900.349071:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.349086:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704ea00, rc 0) 00000100:00100000:0.0:1787784900.349163:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+19:33514:x1874627658339200:12345-192.168.203.43@tcp:101:cp.0 Request processed in 7110us (7650us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784900.349173:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 481 00000100:00100000:2.0:1787784900.353568:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.353697:0:11303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11303:1874627668458368:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784900.353733:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784900.353816:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c94050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2357:x1874627658339328:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5667us (6225us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.353830:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 465 00000100:00100000:0.0:1787784900.354019:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658339584 00000100:00100000:0.0:1787784900.354028:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658339584 00000100:00100000:0.0:1787784900.354085:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 339 00000100:00100000:0.0:1787784900.354095:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+86:33514:x1874627658339584:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:0.0:1787784900.355936:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33514:x1874627658339584:12345-192.168.203.43@tcp:35:cp.0 Request processed in 1831us (2373us total) trans 4294968372 rc 0/0 00000100:00100000:0.0:1787784900.355952:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 339 00000100:00100000:2.0:1787784900.424728:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.425000:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658339712 00000100:00100000:1.0:1787784900.425014:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658339712 00000100:00100000:1.0:1787784900.425097:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1609 00000100:00100000:1.0:1787784900.425111:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+115:33517:x1874627658339712:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784900.425249:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.425277:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8603 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d5e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.425581:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.425602:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.425616:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.425634:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.425641:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.425648:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426003:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8618 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426014:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8618 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426022:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8618 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784900.426065:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x34:0x0]: returning size 158 00000004:00200000:1.0:1787784900.426069:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784900.426092:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8618 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426105:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8618 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426114:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426125:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426369:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426378:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.426383:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426392:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426407:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.426523:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.426529:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426536:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc860a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426553:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426559:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426566:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426571:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.426574:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426581:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426589:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff98951b73b500) 00010000:00010000:1.0:1787784900.426592:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.426595:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426601:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8611 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426646:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8603 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d5e expref: 117 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426895:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8618 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d5e expref: 116 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.426905:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:1.0:1787784900.426907:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784900.426911:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df400, rc 0) 00000100:00100000:1.0:1787784900.427008:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33517:x1874627658339712:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1900us (2296us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.427019:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1609 00000100:00100000:2.0:1787784900.444907:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.444980:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.445065:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658339968 00000100:00100000:1.0:1787784900.445079:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658339968 00000100:00100000:1.0:1787784900.445196:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 482 00000100:00100000:1.0:1787784900.445209:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658339968:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784900.445233:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.445259:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc861f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d9d expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784900.445359:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bed00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668458496:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784900.445485:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4900 going to sleep for 11 seconds 00000100:00080000:2.0:1787784900.446020:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784900.446310:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658339840 00000100:00100000:3.0:1787784900.446318:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658339840 00000100:00100000:3.0:1787784900.446371:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 83 00000100:00100000:3.0:1787784900.446380:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895076cec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:2355:x1874627658339840:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:3.0:1787784900.446452:0:13404:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.446472:0:13404:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc8626 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1d65 expref: -99 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446540:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446557:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446571:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446589:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446596:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446603:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446686:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446695:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446702:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446708:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.446713:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446726:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446741:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.446745:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.446749:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446755:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc862d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446771:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446777:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446795:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446800:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.446803:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446809:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446816:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff9895198b5600) 00010000:00010000:3.0:1787784900.446819:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.446822:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446828:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8634 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446862:0:13404:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc8626 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d65 expref: 117 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446873:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.446876:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784900.446880:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc8626 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d65 expref: 117 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.446891:0:13404:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc -2) 00000100:00100000:3.0:1787784900.447017:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895076cec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:2355:x1874627658339840:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 637us (2126us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784900.447027:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 83 00002000:00010000:0.0:1787784900.487798:0:6671:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1787784900.487811:0:6671:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb mtime: 0 -> 1787784899 00002000:00010000:0.0:1787784900.487815:0:6671:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb atime: 0 -> 1787784899 00002000:00010000:0.0:1787784900.487818:0:6671:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb ctime: 0 -> 1787784899 00000100:00100000:0.0:1787784900.487848:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bed00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668458496:192.168.203.43@tcp:106: 00010000:00010000:0.0:1787784900.487896:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc861f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d9d expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.487939:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc861f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d9d expref: 17 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.487956:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de800, rc 0) 00000100:00100000:0.0:1787784900.488030:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658339968:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 42824us (43048us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784900.488041:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 482 00000100:00100000:2.0:1787784900.497581:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.499220:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658340096 00000100:00100000:3.0:1787784900.499236:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658340096 00000100:00100000:3.0:1787784900.499343:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1610 00000100:00100000:3.0:1787784900.499356:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33517:x1874627658340096:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784900.499382:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.499408:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc863b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1da4 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.499492:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.499508:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.499524:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784900.499598:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:3.0:1787784900.499603:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 7 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784900.499626:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.499667:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc863b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 118 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.499690:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 117 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.499700:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.499703:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784900.499707:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784900.499808:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088512c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33517:x1874627658340096:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 455us (2243us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.499816:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1610 00000100:00100000:2.0:1787784900.513919:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.514041:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658340224 00000100:00100000:1.0:1787784900.514054:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658340224 00000100:00100000:1.0:1787784900.514125:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 483 00000100:00100000:1.0:1787784900.514136:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658340224:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784900.514202:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.514227:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be20fd6d/0xb0ce63137acc8649 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1dab expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784900.514310:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bc780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668458624:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784900.514414:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4900 going to sleep for 11 seconds 00000100:00100000:1.0:1787784900.521685:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bc780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668458624:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784900.521742:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000be20fd6d/0xb0ce63137acc8649 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dab expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.521774:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be20fd6d/0xb0ce63137acc8649 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dab expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.521799:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df200, rc 0) 00000100:00100000:1.0:1787784900.521880:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658340224:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7746us (7976us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784900.521893:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 483 00000100:00100000:2.0:1787784900.527140:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.527288:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658340480 00000100:00100000:0.0:1787784900.527301:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658340480 00000100:00100000:0.0:1787784900.527380:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1611 00000100:00100000:0.0:1787784900.527392:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33517:x1874627658340480:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:0.0:1787784900.527686:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].51, 136 off15128 00000100:00100000:0.0:1787784900.527824:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33517:x1874627658340480:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 433us (697us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784900.527833:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1611 00010000:00010000:3.0:1787784900.529818:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784900.554690:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.554820:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658340608 00000100:00100000:1.0:1787784900.554842:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658340608 00000100:00100000:1.0:1787784900.554922:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1612 00000100:00100000:1.0:1787784900.554935:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33520:x1874627658340608:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784900.554962:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.554989:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8650 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1db9 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555078:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555096:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555110:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555126:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555281:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555290:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555472:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555487:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555495:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555502:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.555508:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555517:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555532:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.555536:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.555541:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555548:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc8657 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555565:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555572:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555579:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555584:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.555588:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555595:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555603:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff98951b73b500) 00010000:00010000:1.0:1787784900.555606:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.555609:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555616:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc865e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555658:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8650 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1db9 expref: 118 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555669:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.555671:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.555675:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc8650 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1db9 expref: 118 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.555684:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df200, rc 0) 00000100:00100000:1.0:1787784900.555771:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33520:x1874627658340608:12345-192.168.203.43@tcp:101:bash.0 Request processed in 839us (1097us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784900.555789:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1612 00000100:00100000:2.0:1787784900.560518:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.560650:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658340736 00000100:00100000:3.0:1787784900.560663:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658340736 00000100:00100000:3.0:1787784900.560729:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 84 00000100:00100000:3.0:1787784900.560741:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:2356:x1874627658340736:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:3.0:1787784900.560801:0:5817:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.560827:0:5817:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc8665 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1db2 expref: -99 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561066:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561086:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561099:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561116:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561123:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561130:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561577:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561587:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561595:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561602:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.561606:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561619:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561634:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.561637:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.561642:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561648:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc866c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561664:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561680:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561687:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561692:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.561695:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561701:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561708:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x71bff0c5 (ffff9895198b5600) 00010000:00010000:3.0:1787784900.561710:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.561713:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561723:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8673 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561762:0:5817:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc8665 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1db2 expref: 118 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561789:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.561792:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784900.561796:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc8665 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1db2 expref: 118 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.561807:0:5817:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc -2) 00000100:00100000:3.0:1787784900.561907:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895088521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:2356:x1874627658340736:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 1167us (1403us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784900.561918:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 84 00000100:00100000:2.0:1787784900.582314:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.583371:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658340864 00000100:00100000:0.0:1787784900.583387:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658340864 00000100:00100000:0.0:1787784900.583479:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1613 00000100:00100000:0.0:1787784900.583494:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33521:x1874627658340864:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784900.583549:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784900.583580:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc867a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1dc0 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.583978:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584350:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584377:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584396:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584403:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584409:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584494:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584502:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584508:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584515:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.584519:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584531:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584546:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.584550:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.584556:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584562:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8681 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584579:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584585:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584591:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584596:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.584599:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584605:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584613:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff989520108a00) 00010000:00010000:0.0:1787784900.584615:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.584618:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584625:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8688 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584692:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc867a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dc0 expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584704:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.584706:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.584710:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc867a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dc0 expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.584719:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fc00, rc 0) 00000100:00100000:0.0:1787784900.584824:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508850000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33521:x1874627658340864:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1334us (2531us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784900.584834:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1613 00000100:00100000:2.0:1787784900.591693:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.593681:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658340992 00000100:00100000:0.0:1787784900.593702:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658340992 00000100:00100000:0.0:1787784900.593885:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1614 00000100:00100000:0.0:1787784900.593900:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33521:x1874627658340992:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784900.593938:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784900.593969:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc868f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1dc7 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594062:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594080:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594096:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594114:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594122:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594129:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594769:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594879:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594886:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594892:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.594896:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594914:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594930:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.594933:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.594939:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594945:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc8696 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594961:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594967:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594972:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594977:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.594980:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594986:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.594992:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff989520108a00) 00010000:00010000:0.0:1787784900.594995:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.594998:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.595004:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc869d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.595053:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc868f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dc7 expref: 88 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.595071:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.595073:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.595077:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc868f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dc7 expref: 88 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.595087:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fc00, rc 0) 00000100:00100000:0.0:1787784900.595234:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33521:x1874627658340992:12345-192.168.203.43@tcp:101:rm.0 Request processed in 1337us (3568us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784900.595244:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1614 00000100:00100000:2.0:1787784900.624588:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.624714:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658341120 00000100:00100000:3.0:1787784900.624728:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658341120 00000100:00100000:3.0:1787784900.624879:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1615 00000100:00100000:3.0:1787784900.624891:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33522:x1874627658341120:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784900.624918:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.624943:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc86a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1dce expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625031:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625048:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625063:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625080:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625088:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625095:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625239:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625248:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625256:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625263:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.625269:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625277:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625291:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.625295:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.625301:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625309:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc86ab lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625324:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625330:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625336:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625342:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.625345:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625351:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625357:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff9895198b5600) 00010000:00010000:3.0:1787784900.625360:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.625363:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625368:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc86b2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625408:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc86a4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dce expref: 88 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625426:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.625429:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.625433:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7600/0xb0ce63137acc86a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dce expref: 88 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.625443:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784900.625541:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402186940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33522:x1874627658341120:12345-192.168.203.43@tcp:101:cp.0 Request processed in 652us (968us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784900.625549:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1615 00000100:00100000:2.0:1787784900.633413:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.634751:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658341248 00000100:00100000:0.0:1787784900.634769:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658341248 00000100:00100000:0.0:1787784900.635383:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1616 00000100:00100000:0.0:1787784900.635397:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33522:x1874627658341248:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784900.635423:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784900.635452:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86b9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1dd5 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635546:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635565:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635580:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635596:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635604:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635611:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635700:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635708:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635716:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635722:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.635728:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635737:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635753:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.635756:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.635762:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635769:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc86c0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635798:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635806:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635812:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635816:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.635819:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635825:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635832:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff989520108a00) 00010000:00010000:0.0:1787784900.635835:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.635838:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635843:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86c7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635904:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86b9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dd5 expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635916:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.635918:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.635921:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86b9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1dd5 expref: 88 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.635938:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fc00, rc 0) 00000100:00100000:0.0:1787784900.636058:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33522:x1874627658341248:12345-192.168.203.43@tcp:101:cp.0 Request processed in 664us (2661us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784900.636067:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1616 00000100:00100000:2.0:1787784900.641756:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.641832:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658341504 00000100:00100000:0.0:1787784900.641846:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658341504 00000100:00100000:0.0:1787784900.642122:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1617 00000100:00100000:0.0:1787784900.642135:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+87:33522:x1874627658341504:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784900.642207:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784900.642234:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc86ce lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ddc expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.642584:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.642605:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 116 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.642615:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.642662:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 116 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.642679:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 116 pid: 20438 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.642702:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668458752:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784900.642850:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787784900.655579:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539102580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668458752:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784900.655641:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784900.661109:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.661696:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658341632 00000100:00100000:1.0:1787784900.661710:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658341632 00010000:00010000:1.0:1787784900.661763:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 117 pid: 20438 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784900.661830:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 466 00000100:00100000:1.0:1787784900.661840:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:2357:x1874627658341632:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784900.661873:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784900.661882:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 117 pid: 20438 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.661898:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 117 pid: 20438 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.661925:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8642 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1da4 expref: 117 pid: 20438 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.661946:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.661952:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.661962:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.661970:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.662006:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784900.662077:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:2357:x1874627658341632:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 239us (986us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.662085:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 466 00010000:00010000:0.0:1787784900.663013:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.663027:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.663058:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.663071:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.663081:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784900.664280:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x23:0x0], index 1: 0 00010000:00010000:0.0:1787784900.665050:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86e3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665072:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665085:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665130:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665138:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665690:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665702:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.665707:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665716:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665731:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.665735:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.665740:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.665747:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc86d5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784900.665765:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001e127746 reply state 00000000b6dee420 transno 4294968373 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.666078:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784900.666129:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc86ce lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ddc expref: 90 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.666998:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fe00/0xb0ce63137acc86e3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ddc expref: 89 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.667012:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff989520108100) 00010000:00010000:0.0:1787784900.667016:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784900.667019:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fe00, rc 0) 00000100:00100000:0.0:1787784900.667539:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33522:x1874627658341504:12345-192.168.203.43@tcp:101:cp.0 Request processed in 25404us (25797us total) trans 4294968373 rc 0/0 00000100:00100000:0.0:1787784900.667550:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1617 00010000:00010000:1.0:1787784900.677017:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.677040:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.677049:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.677056:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.677062:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.677071:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.677092:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff989520108a00) 00010000:00010000:1.0:1787784900.677096:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.677100:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.677107:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc86dc lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784900.682586:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.684793:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658341760 00000100:00100000:1.0:1787784900.684812:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658341760 00000100:00100000:1.0:1787784900.684911:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 85 00000100:00100000:1.0:1787784900.684922:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+89:2356:x1874627658341760:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784900.685019:0:13404:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.685046:0:13404:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc86f1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1de3 expref: -99 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685126:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685143:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685197:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685216:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685223:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685231:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685321:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685330:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685337:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685345:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.685350:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685370:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685385:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.685388:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.685394:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685401:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc86f8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685416:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685422:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685429:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685433:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfe00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.685435:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685441:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685448:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff98951b73b500) 00010000:00010000:1.0:1787784900.685451:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.685453:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685460:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfe00/0xb0ce63137acc86ff lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685499:0:13404:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc86f1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1de3 expref: 90 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685511:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.685513:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784900.685517:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df200/0xb0ce63137acc86f1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1de3 expref: 90 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.685527:0:13404:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df200, rc -2) 00000100:00100000:1.0:1787784900.685621:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539102580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+89:2356:x1874627658341760:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 701us (3050us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784900.685630:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 85 00000100:00100000:2.0:1787784900.688212:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.692030:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658341888 00000100:00100000:3.0:1787784900.692051:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658341888 00000100:00100000:3.0:1787784900.692132:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 484 00000100:00100000:3.0:1787784900.692143:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+17:2356:x1874627658341888:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784900.692190:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.692217:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e1b expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784900.692250:0:13825:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x23:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1787784900.692275:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1787784900.692280:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784900.692291:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 18 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.692322:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 18 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.692343:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784900.692447:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2356:x1874627658341888:12345-192.168.203.43@tcp:101:cp.0 Request processed in 305us (4247us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.692455:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 484 00000100:00100000:2.0:1787784900.702620:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.702760:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658342016 00000100:00100000:1.0:1787784900.702784:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658342016 00000100:00100000:1.0:1787784900.702851:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00100000:1.0:1787784900.703067:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+18:33522:x1874627658342016:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784900.703095:0:22706:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.703121:0:22706:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e22 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.703204:0:22706:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 19 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.703223:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784900.703227:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784900.703233:0:22706:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 19 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.703273:0:22706:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 19 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.703288:0:22706:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 19 pid: 13825 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784900.703511:0:22706:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bc780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668458880:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784900.704072:0:22706:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4240 going to sleep for 11 seconds 00000100:00100000:0.0:1787784900.721809:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.722379:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658342144 00000100:00100000:3.0:1787784900.722392:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658342144 00010000:00010000:3.0:1787784900.722445:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 20 pid: 13825 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.722500:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 467 00000100:00100000:3.0:1787784900.722511:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658342144:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784900.722530:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784900.722559:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 20 pid: 13825 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.722576:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 20 pid: 13825 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.722607:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784900.722611:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784900.722621:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 20 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.722632:0:21579:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 20 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.722675:0:21579:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 20 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.722691:0:21579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:21579:1874627668459008:192.168.203.43@tcp:105: 00000100:00100000:3.0:1787784900.722982:0:21579:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784900.727377:0:21579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef0c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:21579:1874627668459008:192.168.203.43@tcp:105: 00010000:00010000:3.0:1787784900.727414:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784900.727481:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658342144:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 4970us (5685us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.727494:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 467 00010000:00010000:2.0:1787784900.729795:0:22706:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8706 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e1b expref: 19 pid: 13825 timeout: 791 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784900.729827:0:22706:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bc780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668458880:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784900.729862:0:22706:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 18 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.729889:0:22706:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df200, rc 0) 00000100:00100000:2.0:1787784900.729963:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+18:33522:x1874627658342016:12345-192.168.203.43@tcp:101:cp.0 Request processed in 27102us (27358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784900.729978:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00100000:0.0:1787784900.738035:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.738994:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658342400 00000100:00100000:2.0:1787784900.739010:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658342400 00000100:00100000:2.0:1787784900.739095:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:2.0:1787784900.739107:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+89:33522:x1874627658342400:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:2.0:1787784900.739770:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33522:x1874627658342400:12345-192.168.203.43@tcp:35:cp.0 Request processed in 661us (1743us total) trans 4294968374 rc 0/0 00000100:00100000:2.0:1787784900.741035:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 340 00000100:00100000:0.0:1787784900.827686:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.831050:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658342528 00000100:00100000:1.0:1787784900.831072:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658342528 00000100:00100000:1.0:1787784900.831228:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 86 00000100:00100000:1.0:1787784900.831241:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+90:2357:x1874627658342528:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784900.831357:0:5817:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.831555:0:5817:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8714 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e29 expref: -99 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831640:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831658:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831676:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831694:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831702:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831709:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831813:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831821:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831829:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831836:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.831842:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.831851:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.832987:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.832997:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.833006:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833017:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc871b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833041:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833049:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833056:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833062:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.833066:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833073:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833085:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff98951b73b500) 00010000:00010000:1.0:1787784900.833088:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.833091:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833098:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8722 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833214:0:5817:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8714 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e29 expref: 91 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833234:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.833236:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784900.833241:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8714 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e29 expref: 91 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.833251:0:5817:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc -2) 00000100:00100000:0.0:1787784900.836264:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.837655:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+90:2357:x1874627658342528:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 6405us (9971us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784900.837680:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 86 00000100:00100000:3.0:1787784900.837788:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658342656 00000100:00100000:3.0:1787784900.837806:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658342656 00000100:00100000:3.0:1787784900.837876:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1618 00000100:00100000:3.0:1787784900.837886:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+116:33529:x1874627658342656:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784900.837912:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.837938:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8729 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e61 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838023:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838040:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838053:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838069:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838076:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838082:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838238:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc873e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838247:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc873e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838255:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc873e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784900.838294:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x35:0x0]: returning size 158 00000004:00200000:3.0:1787784900.838299:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784900.838770:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc873e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838795:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc873e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838805:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838812:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838824:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838830:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.838835:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838843:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838856:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.838859:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.838871:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838878:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8730 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838895:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838901:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838908:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838912:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.838915:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838925:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838932:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff9895198b5000) 00010000:00010000:3.0:1787784900.838935:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.838938:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838944:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8737 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838978:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8729 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e61 expref: 118 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838992:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7200/0xb0ce63137acc873e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e61 expref: 117 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.838999:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff989520108100) 00010000:00010000:3.0:1787784900.839002:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784900.839005:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7200, rc 0) 00000100:00100000:3.0:1787784900.839088:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33529:x1874627658342656:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 1203us (2835us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784900.839096:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1618 00000100:00100000:0.0:1787784900.851493:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.852431:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658342784 00000100:00100000:2.0:1787784900.852448:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658342784 00000100:00100000:2.0:1787784900.852531:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 486 00000100:00100000:2.0:1787784900.852542:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658342784:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784900.852566:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784900.852590:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005a3b808c/0xb0ce63137acc8745 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e68 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784900.852663:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402185a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668459136:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784900.852797:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a1a9f00 going to sleep for 11 seconds 00002000:00010000:2.0:1787784900.872323:0:6670:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x23:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784900.872336:0:6670:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x23:0x0] updating lvb mtime: 0 -> 1787784900 00002000:00010000:2.0:1787784900.872339:0:6670:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x23:0x0] updating lvb atime: 0 -> 1787784900 00002000:00010000:2.0:1787784900.872341:0:6670:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x23:0x0] updating lvb ctime: 0 -> 1787784900 00000100:00100000:2.0:1787784900.872378:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402185a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668459136:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784900.872429:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005a3b808c/0xb0ce63137acc8745 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e68 expref: 37 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.872467:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a3b808c/0xb0ce63137acc8745 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e68 expref: 37 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784900.872485:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110e00, rc 0) 00000100:00100000:2.0:1787784900.872582:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658342784:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 20040us (21106us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784900.872594:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 486 00000100:00100000:0.0:1787784900.890287:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784900.890649:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658342912 00000100:00100000:3.0:1787784900.890661:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658342912 00000100:00100000:0.0:1787784900.890944:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1619 00000100:00100000:0.0:1787784900.890958:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33529:x1874627658342912:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:0.0:1787784900.890987:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784900.891016:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc874c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e6f expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.891115:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.891132:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.891176:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784900.891261:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:0.0:1787784900.891266:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 8 stripe count 1 MDT count 1 00000004:00010000:0.0:1787784900.891293:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.893004:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc874c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 119 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.893039:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 118 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.893049:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.893052:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787784900.893056:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e800, rc 0) 00000100:00100000:0.0:1787784900.893379:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33529:x1874627658342912:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2423us (3106us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784900.893389:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1619 00000100:00100000:0.0:1787784900.907299:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.907499:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658343040 00000100:00100000:1.0:1787784900.907515:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658343040 00000100:00100000:1.0:1787784900.907594:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 487 00000100:00100000:1.0:1787784900.907605:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658343040:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784900.907630:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.907654:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc875a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e76 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784900.907735:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bf0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668459264:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784900.907859:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989502ab4900 going to sleep for 11 seconds 00000100:00100000:1.0:1787784900.917766:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bf0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668459264:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784900.917836:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc875a lrc: 1/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e76 expref: 37 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.918223:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc875a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e76 expref: 37 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.918244:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00000100:00100000:1.0:1787784900.919066:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658343040:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 11458us (11778us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784900.919078:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 487 00000100:00100000:2.0:1787784900.948652:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.949382:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658343296 00000100:00100000:0.0:1787784900.949397:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658343296 00000100:00100000:0.0:1787784900.949477:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1620 00000100:00100000:0.0:1787784900.949489:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33529:x1874627658343296:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:0.0:1787784900.952287:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].52, 136 off15264 00000100:00100000:1.0:1787784900.957209:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33529:x1874627658343296:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 7705us (8553us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784900.957229:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1620 00000100:00100000:0.0:1787784900.972035:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.972192:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658343424 00000100:00100000:1.0:1787784900.972206:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658343424 00000100:00100000:1.0:1787784900.972276:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1621 00000100:00100000:1.0:1787784900.972287:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33530:x1874627658343424:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:1.0:1787784900.972313:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.972337:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8761 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e7d expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972429:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972447:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972461:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972477:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972489:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972496:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972579:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972587:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972594:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972601:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.972606:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972614:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972628:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.972631:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.972636:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972643:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc8768 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972659:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972666:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972681:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972685:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.972689:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972695:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972702:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff98951b73b500) 00010000:00010000:1.0:1787784900.972704:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.972707:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972712:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc876f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972752:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8761 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e7d expref: 119 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972764:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.972767:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.972784:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8761 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e7d expref: 119 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.972802:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00000100:00100000:1.0:1787784900.972883:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33530:x1874627658343424:12345-192.168.203.43@tcp:101:bash.0 Request processed in 597us (861us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784900.972890:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1621 00000100:00100000:2.0:1787784900.986485:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784900.986547:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784900.986827:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658343552 00000100:00100000:0.0:1787784900.986840:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658343552 00000100:00100000:0.0:1787784900.986931:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1622 00000100:00100000:0.0:1787784900.986942:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33531:x1874627658343552:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:0.0:1787784900.986970:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784900.986997:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8776 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e84 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.987081:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.987098:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.987113:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.987128:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.987135:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.987142:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784900.988076:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658343680 00000100:00100000:3.0:1787784900.988089:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658343680 00000100:00100000:3.0:1787784900.988204:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 87 00000100:00100000:3.0:1787784900.988216:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+92:2356:x1874627658343680:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:3.0:1787784900.988252:0:5816:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784900.988269:0:5816:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc878b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e8b expref: -99 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988312:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988323:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988335:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988347:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988354:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988360:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988446:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988453:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988461:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988471:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.988476:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988484:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988499:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.988502:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.988508:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988514:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8792 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988532:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988538:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988545:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988549:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784900.988551:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988558:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988568:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x21330256 (ffff9895198b5000) 00010000:00010000:3.0:1787784900.988570:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784900.988573:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988580:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc8799 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988609:0:5816:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc878b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e8b expref: 93 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988619:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784900.988621:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784900.988624:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc878b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e8b expref: 93 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784900.988632:0:5816:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6800, rc -2) 00000100:00100000:3.0:1787784900.988722:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402187840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+92:2356:x1874627658343680:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 507us (2173us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787784900.988745:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 87 00010000:00010000:0.0:1787784900.989419:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989427:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989436:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989441:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.989444:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989450:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989462:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.989465:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.989468:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989472:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc877d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989485:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989491:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989496:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989499:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784900.989501:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989509:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989515:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff989520108c00) 00010000:00010000:0.0:1787784900.989517:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.989519:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989523:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8784 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989563:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8776 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e84 expref: 91 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989570:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784900.989572:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784900.989574:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8776 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e84 expref: 91 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784900.989581:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e000, rc 0) 00000100:00100000:0.0:1787784900.989630:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33531:x1874627658343552:12345-192.168.203.43@tcp:101:rm.0 Request processed in 2690us (3162us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784900.989637:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1622 00000100:00100000:2.0:1787784900.996416:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784900.996496:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658343808 00000100:00100000:1.0:1787784900.996511:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658343808 00000100:00100000:1.0:1787784900.996601:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1623 00000100:00100000:1.0:1787784900.996615:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33531:x1874627658343808:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:1.0:1787784900.996646:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784900.996676:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc87a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e92 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996791:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996810:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996825:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996843:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996851:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996859:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996947:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996957:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996965:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996973:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.996978:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.996988:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997003:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.997007:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.997012:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997019:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997240:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997248:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997255:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997262:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784900.997265:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997272:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997282:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff98951b73b500) 00010000:00010000:1.0:1787784900.997284:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.997288:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997295:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997348:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc87a0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e92 expref: 91 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997360:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784900.997363:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784900.997366:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc87a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e92 expref: 91 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784900.997377:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00000100:00100000:1.0:1787784900.997462:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33531:x1874627658343808:12345-192.168.203.43@tcp:101:rm.0 Request processed in 849us (1061us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784900.997472:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1623 00000100:00100000:3.0:1787784901.029422:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.036228:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658343936 00000100:00100000:0.0:1787784901.036249:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658343936 00000100:00100000:0.0:1787784901.036348:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1624 00000100:00100000:0.0:1787784901.036360:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33532:x1874627658343936:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784901.036391:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784901.036418:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87b5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1e99 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036504:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036520:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036534:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036549:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036555:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036562:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036744:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036753:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036761:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036767:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784901.036773:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036792:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036808:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784901.036811:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784901.036817:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036824:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc87bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036840:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036847:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036854:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036860:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784901.036863:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036870:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036877:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff989520108c00) 00010000:00010000:0.0:1787784901.036879:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784901.036883:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036889:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036944:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87b5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e99 expref: 91 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036956:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784901.036959:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784901.036963:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87b5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e99 expref: 91 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.036973:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e000, rc 0) 00000100:00100000:0.0:1787784901.037947:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33532:x1874627658343936:12345-192.168.203.43@tcp:101:cp.0 Request processed in 1579us (8529us total) trans 0 rc 301/301 00000100:00100000:0.0:1787784901.037959:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1624 00000100:00100000:3.0:1787784901.042705:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.042957:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658344064 00000100:00100000:1.0:1787784901.042970:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658344064 00000100:00100000:1.0:1787784901.043092:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1625 00000100:00100000:1.0:1787784901.043104:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33532:x1874627658344064:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784901.043142:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.043204:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc87ca lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ea0 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043292:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043311:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043326:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043341:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043348:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043355:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043447:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043455:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043462:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043470:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.043477:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043485:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043501:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.043504:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.043509:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043516:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc87d1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043536:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043542:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043549:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043555:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.043557:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043562:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043574:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff98951b73b500) 00010000:00010000:1.0:1787784901.043576:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.043579:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043585:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc87d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043620:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc87ca lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ea0 expref: 91 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043631:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.043634:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.043638:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc87ca lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ea0 expref: 91 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.043647:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00000100:00100000:1.0:1787784901.043736:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33532:x1874627658344064:12345-192.168.203.43@tcp:101:cp.0 Request processed in 634us (1046us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784901.043745:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1625 00000100:00100000:2.0:1787784901.063826:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.067435:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658344320 00000100:00100000:3.0:1787784901.067456:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658344320 00000100:00100000:3.0:1787784901.067563:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1626 00000100:00100000:3.0:1787784901.067580:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+90:33532:x1874627658344320:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784901.067608:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.067636:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc87df lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ea7 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.067850:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.067869:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 117 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.067879:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.067929:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 117 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.067945:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 117 pid: 5812 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.067962:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668459392:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784901.068087:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784901.076385:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668459392:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784901.076434:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.080577:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.081039:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658344448 00000100:00100000:3.0:1787784901.081052:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658344448 00010000:00010000:3.0:1787784901.081108:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 118 pid: 5812 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.082066:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 468 00000100:00100000:3.0:1787784901.082088:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:2357:x1874627658344448:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784901.082109:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784901.082120:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 118 pid: 5812 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.082173:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 118 pid: 5812 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.082203:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8753 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e6f expref: 118 pid: 5812 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.082229:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784901.082236:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.082247:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.082256:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.082293:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784901.082395:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:2357:x1874627658344448:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 317us (1835us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.082405:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 468 00010000:00010000:0.0:1787784901.084898:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.084920:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.084975:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.084990:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.085004:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787784901.085722:0:5813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x21:0x0], index 0: 0 00010000:00010000:0.0:1787784901.085957:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87f4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.085970:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.085980:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086020:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086028:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086035:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086043:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784901.086048:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086056:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086071:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784901.086074:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784901.086079:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086091:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6600/0xb0ce63137acc87e6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784901.086107:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000075c60a53 reply state 000000009a036f7b transno 4294968375 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.086118:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787784901.086200:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc87df lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ea7 expref: 93 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086216:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f400/0xb0ce63137acc87f4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ea7 expref: 92 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.086224:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff989520108e00) 00010000:00010000:0.0:1787784901.086226:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784901.086229:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704f400, rc 0) 00000100:00100000:0.0:1787784901.086323:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+93:33532:x1874627658344320:12345-192.168.203.43@tcp:101:cp.0 Request processed in 18749us (22510us total) trans 4294968375 rc 0/0 00000100:00100000:0.0:1787784901.086333:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1626 00010000:00010000:3.0:1787784901.102633:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.102656:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.102666:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.102676:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.102682:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.102691:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.102712:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff989520108c00) 00010000:00010000:3.0:1787784901.102715:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.102721:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.102730:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc87ed lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.103524:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.103800:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658344576 00000100:00100000:0.0:1787784901.103813:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658344576 00000100:00100000:0.0:1787784901.103888:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 488 00000100:00100000:0.0:1787784901.103900:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2357:x1874627658344576:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:0.0:1787784901.103924:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784901.103948:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1eae expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787784901.103988:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x21:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1787784901.104012:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1787784901.104018:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784901.104030:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 19 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.104051:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 19 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.104065:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e400, rc 0) 00000100:00100000:0.0:1787784901.104140:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2357:x1874627658344576:12345-192.168.203.43@tcp:101:cp.0 Request processed in 242us (632us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.104164:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 488 00000100:00100000:2.0:1787784901.107266:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.113034:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658344704 00000100:00100000:3.0:1787784901.113055:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658344704 00000100:00100000:3.0:1787784901.113167:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 489 00000100:00100000:3.0:1787784901.113180:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+19:33532:x1874627658344704:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784901.113205:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.113227:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1eb5 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.113269:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 20 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.113280:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784901.113285:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784901.113293:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 20 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.113340:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 20 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.113354:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 20 pid: 6671 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.113388:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030efc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668459520:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784901.113503:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169600 going to sleep for 11 seconds 00000100:00100000:3.0:1787784901.123701:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030efc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668459520:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784901.123749:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 20 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.123785:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784901.125933:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+20:33532:x1874627658344704:12345-192.168.203.43@tcp:101:cp.0 Request processed in 12737us (18663us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.125950:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 489 00000100:00100000:0.0:1787784901.126436:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.127573:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658344832 00000100:00100000:3.0:1787784901.127588:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658344832 00010000:00010000:0.0:1787784901.127730:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 20 pid: 6671 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.127843:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00100000:0.0:1787784901.127854:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658344832:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784901.127872:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784901.127901:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 20 pid: 6671 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.127919:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 20 pid: 6671 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.127949:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009af7738c/0xb0ce63137acc8802 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eae expref: 20 pid: 6671 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.127969:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784901.127974:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784901.127984:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 19 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.127997:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 19 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.128045:0:5795:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 19 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.128064:0:5795:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b45de00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668459648:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784901.128211:0:5795:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787784901.132567:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.132906:0:5795:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b45de00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5795:1874627668459648:192.168.203.43@tcp:105: 00010000:00010000:1.0:1787784901.132955:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784901.133093:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2357:x1874627658344832:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5239us (6657us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.133108:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 469 00000100:00100000:2.0:1787784901.133124:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658345088 00000100:00100000:2.0:1787784901.133145:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658345088 00000100:00100000:2.0:1787784901.133645:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:2.0:1787784901.133656:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+92:33532:x1874627658345088:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:2.0:1787784901.134081:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+93:33532:x1874627658345088:12345-192.168.203.43@tcp:35:cp.0 Request processed in 425us (1528us total) trans 4294968376 rc 0/0 00000100:00100000:2.0:1787784901.134090:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 341 00000100:00100000:0.0:1787784901.150714:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.150907:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658345216 00000100:00100000:1.0:1787784901.150920:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658345216 00000100:00100000:1.0:1787784901.150988:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 88 00000100:00100000:1.0:1787784901.150999:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+93:2357:x1874627658345216:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784901.151102:0:5817:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.151128:0:5817:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8810 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ebc expref: -99 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151243:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151271:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151284:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151302:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151310:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151318:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151404:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151417:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151425:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151432:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.151437:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151446:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151461:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.151469:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.151475:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151483:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8817 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151505:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151512:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151520:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151526:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.151531:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151538:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151545:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x21330256 (ffff98951b73b500) 00010000:00010000:1.0:1787784901.151548:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.151551:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151557:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc881e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x21330256 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151593:0:5817:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8810 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ebc expref: 94 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151604:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.151607:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784901.151611:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8810 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ebc expref: 94 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.151621:0:5817:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc -2) 00000100:00100000:1.0:1787784901.151727:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+93:2357:x1874627658345216:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 729us (1026us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784901.151736:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 88 00000040:00080000:2.0:1787784901.171352:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784901.171374:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784901.171447:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 53 count 11 00000040:00080000:2.0:1787784901.171455:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784901.172886:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adc780 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668459776:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784901.173082:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be300 going to sleep for 11 seconds 00000100:00100000:0.0:1787784901.181013:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.181267:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658345344 00000100:00100000:1.0:1787784901.181280:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658345344 00000100:00100000:1.0:1787784901.181601:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1627 00000100:00100000:1.0:1787784901.181614:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33535:x1874627658345344:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784901.181641:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.181669:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8825 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ec3 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181749:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181782:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181797:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181816:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181824:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181831:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181950:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc883a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181959:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc883a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.181967:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc883a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784901.182006:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x36:0x0]: returning size 158 00000004:00200000:1.0:1787784901.182012:0:10850:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1787784901.182033:0:10850:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc883a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182047:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc883a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182057:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182065:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182072:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182079:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.182084:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182092:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182104:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.182108:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.182113:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182119:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc882c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182144:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182210:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182215:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182220:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.182223:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182229:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182235:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff98951b73b500) 00010000:00010000:1.0:1787784901.182237:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.182239:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182245:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8833 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182277:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8825 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ec3 expref: 119 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182288:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dea00/0xb0ce63137acc883a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ec3 expref: 118 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.182294:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff989520108e00) 00010000:00010000:1.0:1787784901.182296:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784901.182308:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dea00, rc 0) 00000100:00100000:1.0:1787784901.182608:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33535:x1874627658345344:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 769us (1382us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.182618:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1627 00000100:00100000:1.0:1787784901.187751:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adc780 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668459776:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1787784901.197404:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.197464:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.197492:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.198255:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658345728 00000100:00100000:1.0:1787784901.198268:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658345728 00000100:00100000:1.0:1787784901.198678:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1628 00000100:00100000:1.0:1787784901.198691:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+93:33538:x1874627658345728:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784901.198737:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x33:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784901.199052:0:13404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658345600 00000100:00100000:2.0:1787784901.199059:0:13404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658345600 00000100:00100000:2.0:1787784901.199104:0:13404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 89 00000100:00100000:2.0:1787784901.199113:0:13404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:2354:x1874627658345600:12345-192.168.203.43@tcp:63:ll_sa.0 00000100:00100000:1.0:1787784901.199130:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+93:33538:x1874627658345728:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 440us (1635us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.199142:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1628 00010000:00010000:2.0:1787784901.199194:0:13404:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.199224:0:13404:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8841 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ed1 expref: -99 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199295:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199314:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199339:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199366:0:13404:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199376:0:13404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199383:0:13404:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199464:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199473:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199480:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199487:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.199492:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199501:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199515:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784901.199518:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.199523:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199530:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc8848 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199544:0:13404:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199550:0:13404:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199556:0:13404:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199562:0:13404:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.199565:0:13404:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199572:0:13404:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199579:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff989520131600) 00010000:00010000:2.0:1787784901.199582:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.199586:0:13404:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199592:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc884f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13404 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199626:0:13404:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8841 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ed1 expref: 119 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199637:0:13404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784901.199640:0:13404:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787784901.199643:0:13404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8841 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ed1 expref: 119 pid: 13404 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.199653:0:13404:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110a00, rc -2) 00000100:00100000:2.0:1787784901.199690:0:13404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:2354:x1874627658345600:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 579us (2226us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787784901.199698:0:13404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 89 00000100:00100000:2.0:1787784901.199733:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658345472 00000100:00100000:2.0:1787784901.199738:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658345472 00000100:00100000:2.0:1787784901.199792:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 490 00000100:00100000:2.0:1787784901.199800:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658345472:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784901.199812:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.199825:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc8856 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1eca expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.200104:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.201786:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658345856 00000100:00100000:1.0:1787784901.201801:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658345856 00000100:00100000:1.0:1787784901.201869:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1629 00000100:00100000:1.0:1787784901.201880:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+93:33539:x1874627658345856:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784901.201911:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x34:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784901.202029:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+93:33539:x1874627658345856:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 149us (1925us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.202037:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1629 00000100:00100000:2.0:1787784901.202674:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668459904:192.168.203.43@tcp:106: 00000100:00100000:2.0:1787784901.202758:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be840 going to sleep for 11 seconds 00000100:00100000:3.0:1787784901.212287:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.212598:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658346112 00000100:00100000:1.0:1787784901.212613:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658346112 00000100:00100000:1.0:1787784901.212698:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1630 00000100:00100000:1.0:1787784901.212710:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+93:33538:x1874627658346112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784901.212738:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.212782:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc885d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f09 expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.212970:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8864 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.212988:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8864 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.213002:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8864 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787784901.213017:0:10850:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 158 00010000:00010000:1.0:1787784901.213255:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc885d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f09 expref: 96 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.213278:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8864 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f09 expref: 95 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.213289:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784901.213293:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787784901.213297:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df000, rc 0) 00000100:00100000:1.0:1787784901.213387:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+95:33538:x1874627658346112:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 679us (1116us total) trans 4294968377 rc 0/0 00000100:00100000:1.0:1787784901.213396:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1630 00000100:00100000:0.0:1787784901.216328:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.217586:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658346240 00000100:00100000:1.0:1787784901.217601:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658346240 00000100:00100000:1.0:1787784901.217693:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1631 00000100:00100000:1.0:1787784901.217706:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+95:33540:x1874627658346240:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1787784901.217753:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1787784901.218119:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+95:33540:x1874627658346240:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 414us (1803us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.218128:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1631 00000100:00100000:2.0:1787784901.228578:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.228759:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658346496 00000100:00100000:3.0:1787784901.228786:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658346496 00000100:00100000:3.0:1787784901.229256:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1632 00000100:00100000:3.0:1787784901.229270:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+95:33539:x1874627658346496:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1787784901.229305:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.229744:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8872 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f10 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.230272:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8879 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.230291:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8879 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.230304:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8879 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784901.230318:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x34:0x0]: returning size 158 00000100:00100000:0.0:1787784901.230544:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:3.0:1787784901.230720:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8872 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f10 expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.230745:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8879 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f10 expref: 97 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.230755:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:3.0:1787784901.230759:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787784901.230764:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784901.230874:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33539:x1874627658346496:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1607us (2310us total) trans 4294968378 rc 0/0 00000100:00100000:3.0:1787784901.230889:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1632 00000100:00100000:2.0:1787784901.231208:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.231381:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.231417:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658346752 00000100:00100000:2.0:1787784901.231425:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658346752 00000100:00100000:2.0:1787784901.231481:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 73 00000100:00100000:2.0:1787784901.231491:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+97:2356:x1874627658346752:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784901.231506:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784901.231528:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8864 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f09 expref: 97 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.231543:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:2.0:1787784901.231547:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784901.231571:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+96:2356:x1874627658346752:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 82us (190us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.231577:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 73 00002000:00010000:2.0:1787784901.231688:0:6669:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1787784901.231692:0:6669:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb mtime: 0 -> 1787784900 00002000:00010000:2.0:1787784901.231695:0:6669:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb atime: 0 -> 1787784900 00002000:00010000:2.0:1787784901.231697:0:6669:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb ctime: 0 -> 1787784900 00000100:00100000:2.0:1787784901.231717:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6669:1874627668459904:192.168.203.43@tcp:106: 00010000:00010000:2.0:1787784901.231747:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc8856 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eca expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.231784:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc8856 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eca expref: 17 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.231795:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111000, rc 0) 00000100:00100000:2.0:1787784901.231824:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2357:x1874627658345472:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 32024us (34437us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784901.231832:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 490 00000100:00100000:2.0:1787784901.231847:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658346624 00000100:00100000:2.0:1787784901.231850:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658346624 00000100:00100000:2.0:1787784901.232200:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 491 00000100:00100000:2.0:1787784901.232211:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+18:2356:x1874627658346624:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.232258:0:22706:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.232278:0:22706:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 18 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.232287:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x22:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784901.232292:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784901.232297:0:22706:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.232340:0:22706:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 18 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.232413:0:22706:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 18 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.232434:0:22706:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402ade1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668460032:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784901.232508:0:22706:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be840 going to sleep for 11 seconds 00000100:00100000:2.0:1787784901.237447:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.237497:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347136 00000100:00100000:2.0:1787784901.237512:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347136 00000100:00100000:2.0:1787784901.237583:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 492 00000100:00100000:2.0:1787784901.237595:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2357:x1874627658347136:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.237658:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.237683:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.237693:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x20:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787784901.237697:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784901.237703:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 4/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.237748:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.237764:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 19 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.237797:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adc3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668460160:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784901.238254:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033bee40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784901.238300:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.238572:0:22706:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402ade1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668460032:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784901.238600:0:22706:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.238652:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.241136:0:18489:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347392 00000100:00100000:2.0:1787784901.241173:0:18489:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347392 00002000:00100000:2.0:1787784901.241253:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x22:0x0] (0->157) req@ffff989402ade1c0 x1874627658347392/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784901.241265:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402ade1c0 with x1874627658347392 ext(0->157) 00010000:00010000:2.0:1787784901.241275:0:18489:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 19 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.241292:0:18489:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 19 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784901.241300:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989402ade1c0 00000100:00100000:2.0:1787784901.241347:0:18489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 22 00000100:00100000:2.0:1787784901.241370:0:18489:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2354:x1874627658347392:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784901.242182:0:18489:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2354:x1874627658347392:12345-192.168.203.43@tcp:4:cp.0 Request processed in 820us (3525us total) trans 4294967393 rc 0/0 00000100:00100000:2.0:1787784901.242193:0:18489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 22 00002000:00100000:2.0:1787784901.242994:0:18489:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x22:0x0] (0->157) req@ffff989402ade1c0 x1874627658347392/t4294967393(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/448 e 0 to 0 dl 1787784912 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784901.243008:0:18489:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402ade1c0 with x1874627658347392 ext(0->157) 00010000:00010000:2.0:1787784901.243020:0:18489:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 21 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.243034:0:18489:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 21 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784901.243043:0:18489:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989402ade1c0 00000100:00100000:2.0:1787784901.243109:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347264 00000100:00100000:2.0:1787784901.243116:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347264 00000100:00100000:2.0:1787784901.243203:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 74 00000100:00100000:2.0:1787784901.243213:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+96:2355:x1874627658347264:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784901.243227:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784901.243256:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8879 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f10 expref: 96 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.243278:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784901.243284:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784901.244525:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+95:2355:x1874627658347264:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1299us (6211us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.244541:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 74 00000100:00100000:2.0:1787784901.246352:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.246403:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347008 00000100:00100000:2.0:1787784901.246414:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347008 00000100:00100000:2.0:1787784901.246473:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347520 00000100:00100000:2.0:1787784901.246476:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347520 00000100:00100000:2.0:1787784901.246497:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1633 00000100:00100000:2.0:1787784901.246508:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+95:33540:x1874627658347008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.246537:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.246566:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8895 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f17 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.246617:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1634 00000100:00100000:3.0:1787784901.246628:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33535:x1874627658347520:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784901.246636:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.246647:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc889c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f1e expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.246709:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc88a3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.246727:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc88a3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.246741:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc88a3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784901.246755:0:5813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x35:0x0]: returning size 158 00010000:00010000:3.0:1787784901.246840:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.246850:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.246859:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784901.247517:0:5811:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:3.0:1787784901.247522:0:5811:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 9 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784901.247551:0:5811:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.247730:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc889c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 120 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.247934:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 119 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.247945:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784901.247949:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784901.247953:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00010000:00010000:2.0:1787784901.248037:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110a00/0xb0ce63137acc8895 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f17 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.248053:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc88a3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f17 expref: 97 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.248061:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff989520108100) 00010000:00010000:2.0:1787784901.248064:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784901.248067:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111600, rc 0) 00000100:00100000:2.0:1787784901.248140:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33540:x1874627658347008:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1632us (17599us total) trans 4294968379 rc 0/0 00000100:00100000:2.0:1787784901.248168:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1633 00000100:00100000:2.0:1787784901.248312:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adc3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668460160:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784901.248351:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.248644:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:33535:x1874627658347520:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2017us (2301us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.248651:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1634 00000100:00100000:2.0:1787784901.249050:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.249233:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.249563:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347776 00000100:00100000:1.0:1787784901.249573:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347776 00010000:00010000:1.0:1787784901.249618:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 21 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.249649:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 470 00000100:00100000:1.0:1787784901.249660:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530c96850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2357:x1874627658347776:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784901.252638:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784901.252711:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 21 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.252734:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 21 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.252784:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003b43e5e2/0xb0ce63137acc84ac lrc: 0/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cd9 expref: 21 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.252807:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784901.252812:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784901.252819:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.252838:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.252896:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784901.252976:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530c96850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658347776:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 3315us (3740us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.252990:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 470 00010000:00010000:0.0:1787784901.254353:0:22706:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254381:0:22706:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254423:0:22706:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254431:0:22706:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254440:0:22706:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254448:0:22706:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784901.254453:0:22706:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254463:0:22706:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254483:0:22706:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.254491:0:22706:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc8887 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.254613:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2356:x1874627658346624:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 22398us (23399us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.254629:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 491 00000100:00100000:3.0:1787784901.255276:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347648 00000100:00100000:3.0:1787784901.255290:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347648 00002000:00100000:1.0:1787784901.255511:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x20:0x0] (0->157) req@ffff989402adc3c0 x1874627658347648/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.255524:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402adc3c0 with x1874627658347648 ext(0->157) 00010000:00010000:1.0:1787784901.255533:0:6676:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 20 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.255550:0:6676:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 20 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.255558:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989402adc3c0 00000100:00100000:1.0:1787784901.255591:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 23 00000100:00100000:1.0:1787784901.255602:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658347648:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784901.258732:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658347648:12345-192.168.203.43@tcp:4:cp.0 Request processed in 3112us (9663us total) trans 4294967379 rc 0/0 00000100:00100000:1.0:1787784901.258752:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 23 00002000:00100000:1.0:1787784901.258784:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x20:0x0] (0->157) req@ffff989402adc3c0 x1874627658347648/t4294967379(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/448 e 0 to 0 dl 1787784912 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.258802:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402adc3c0 with x1874627658347648 ext(0->157) 00010000:00010000:1.0:1787784901.259078:0:6676:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 22 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.259099:0:6676:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 22 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.259109:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989402adc3c0 00000100:00100000:0.0:1787784901.259225:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.259286:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658347904 00000100:00100000:1.0:1787784901.259294:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658347904 00000100:00100000:1.0:1787784901.259341:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 493 00000100:00100000:1.0:1787784901.259349:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658347904:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784901.259373:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.259395:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e223e7e/0xb0ce63137acc88b8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f25 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.259455:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bf0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668460288:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784901.260648:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784901.261344:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.261412:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.261479:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.261517:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658348416 00000100:00100000:2.0:1787784901.261525:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658348416 00000100:00100000:2.0:1787784901.261575:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1635 00000100:00100000:2.0:1787784901.261585:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33538:x1874627658348416:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784901.261759:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33538:x1874627658348416:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 174us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.261767:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1635 00000100:00100000:3.0:1787784901.262105:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658348160 00000100:00100000:3.0:1787784901.262110:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658348160 00000100:00100000:3.0:1787784901.262173:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 75 00000100:00100000:3.0:1787784901.262181:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+97:2354:x1874627658348160:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784901.262198:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784901.262226:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc88a3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f17 expref: 97 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.262247:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff989520108100) 00010000:00010000:3.0:1787784901.262254:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784901.262289:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+96:2354:x1874627658348160:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 109us (876us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.262297:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 75 00000100:00100000:0.0:1787784901.262575:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658348032 00000100:00100000:0.0:1787784901.262580:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658348032 00000100:00100000:0.0:1787784901.262654:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 494 00000100:00100000:0.0:1787784901.262664:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+19:2356:x1874627658348032:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784901.262703:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/1,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.262724:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 19 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.262734:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x23:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787784901.262739:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787784901.262745:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 4/1,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.262796:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 19 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.262810:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 19 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.262828:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668460416:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784901.262891:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1fe40 going to sleep for 11 seconds 00000100:00100000:2.0:1787784901.263728:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.263954:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bf0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668460288:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784901.264641:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658348544 00000100:00100000:1.0:1787784901.264651:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658348544 00010000:00010000:1.0:1787784901.264687:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 22 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.264714:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e223e7e/0xb0ce63137acc88b8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f25 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.264726:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 471 00000100:00100000:1.0:1787784901.264736:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2356:x1874627658348544:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784901.264740:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e223e7e/0xb0ce63137acc88b8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f25 expref: 17 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.264752:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784901.264770:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 22 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.264796:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 22 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.264815:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d77e71e4/0xb0ce63137acc85fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d57 expref: 22 pid: 6669 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.264822:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00010000:00010000:1.0:1787784901.264827:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784901.264831:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784901.264836:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.264847:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.265088:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784901.265120:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2356:x1874627658348544:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 384us (1394us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.265128:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 471 00000100:00100000:3.0:1787784901.266481:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2356:x1874627658347904:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 7123us (7247us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784901.266494:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 493 00010000:00010000:1.0:1787784901.267548:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267567:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267609:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267617:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267626:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267633:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784901.267638:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267648:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267667:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.267688:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc888e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.267788:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658347136:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 30192us (30353us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.267800:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 492 00000100:00100000:2.0:1787784901.268718:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6671:1874627668460416:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784901.268764:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/1,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.270766:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.271608:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.271917:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658348800 00000100:00100000:1.0:1787784901.271930:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658348800 00002000:00100000:1.0:1787784901.272524:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x23:0x0] (0->157) req@ffff989404dbda40 x1874627658348800/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.272537:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbda40 with x1874627658348800 ext(0->157) 00010000:00010000:1.0:1787784901.272550:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 20 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.272571:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 20 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.272579:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989404dbda40 00000100:00100000:1.0:1787784901.272618:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 24 00000100:00100000:1.0:1787784901.272632:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2357:x1874627658348800:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:3.0:1787784901.273812:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.273869:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.273995:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658349056 00000100:00100000:0.0:1787784901.274005:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658349056 00000100:00100000:0.0:1787784901.274059:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658349312 00000100:00100000:0.0:1787784901.274063:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658349312 00000100:00100000:0.0:1787784901.274093:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1636 00000100:00100000:0.0:1787784901.274103:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:33535:x1874627658349056:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:0.0:1787784901.274394:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].53, 136 off15400 00000100:00100000:0.0:1787784901.274509:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:33535:x1874627658349056:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 407us (705us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.274518:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1636 00000100:00100000:0.0:1787784901.274548:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1637 00000100:00100000:0.0:1787784901.274555:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+96:33539:x1874627658349312:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1787784901.274602:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+96:33539:x1874627658349312:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 47us (741us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.274608:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1637 00000100:00100000:1.0:1787784901.275373:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658348800:12345-192.168.203.43@tcp:4:cp.0 Request processed in 2740us (3763us total) trans 4294967394 rc 0/0 00000100:00100000:1.0:1787784901.275384:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 24 00002000:00100000:1.0:1787784901.275396:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x23:0x0] (0->157) req@ffff989404dbda40 x1874627658348800/t4294967394(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/448 e 0 to 0 dl 1787784912 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.275408:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404dbda40 with x1874627658348800 ext(0->157) 00010000:00010000:1.0:1787784901.275418:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 22 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.275443:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 22 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.275452:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff989404dbda40 00000100:00100000:2.0:1787784901.278249:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658348672 00000100:00100000:2.0:1787784901.278269:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658348672 00000100:00100000:2.0:1787784901.278587:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 495 00000100:00100000:2.0:1787784901.278601:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2356:x1874627658348672:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.278624:0:22706:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.278797:0:22706:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f2c expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784901.278941:0:22706:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x22:0x0] initial LVB size: 158, mtime: 0x6a8f6ec3, atime: 0x0, ctime: 0x6a8f6ec3, blocks: 0x1 00010000:00010000:2.0:1787784901.278969:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1787784901.278975:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.278987:0:22706:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 23 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.279145:0:22706:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 23 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.279320:0:22706:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110a00, rc 0) 00000100:00100000:2.0:1787784901.279517:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2356:x1874627658348672:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 919us (8762us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.279634:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 495 00000100:00100000:0.0:1787784901.287956:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.287993:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.288034:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658349440 00000100:00100000:0.0:1787784901.288037:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.288047:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658349440 00000100:00100000:2.0:1787784901.288120:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 496 00000100:00100000:2.0:1787784901.288130:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+20:2356:x1874627658349440:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.288194:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.288222:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f33 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784901.288266:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x20:0x0] initial LVB size: 158, mtime: 0x6a8f6ec3, atime: 0x0, ctime: 0x6a8f6ec3, blocks: 0x1 00010000:00010000:2.0:1787784901.288293:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1787784901.288298:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.288310:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 21 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.288329:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 21 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.288346:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110400, rc 0) 00000100:00100000:2.0:1787784901.288415:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2356:x1874627658349440:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 286us (471us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.288424:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 496 00000100:00100000:3.0:1787784901.288680:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658349568 00000100:00100000:3.0:1787784901.288687:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658349568 00010000:00010000:3.0:1787784901.288718:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 23 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.288751:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 472 00000100:00100000:3.0:1787784901.288759:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2357:x1874627658349568:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784901.288771:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784901.288800:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 23 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.288813:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 23 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.288837:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc870d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1e22 expref: 23 pid: 22706 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.288853:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784901.288857:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787784901.288862:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.288871:0:21579:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.288905:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784901.288934:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658349568:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 176us (940us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.288942:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 472 00010000:00010000:0.0:1787784901.289031:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289042:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289073:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289083:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289091:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289100:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ee00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784901.289105:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289114:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289128:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 2/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.289136:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b14a4357/0xb0ce63137acc88bf lrc: 0/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.289236:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2356:x1874627658348032:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 26574us (27898us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.289245:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 494 00000100:00100000:1.0:1787784901.289600:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658349824 00000100:00100000:1.0:1787784901.289606:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658349824 00000100:00100000:1.0:1787784901.289651:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1638 00000100:00100000:1.0:1787784901.289660:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+96:33538:x1874627658349824:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784901.289735:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+96:33538:x1874627658349824:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 76us (1696us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.289743:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1638 00000100:00100000:0.0:1787784901.296266:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.296367:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.296658:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658350080 00000100:00100000:2.0:1787784901.296668:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658350080 00000100:00100000:2.0:1787784901.296736:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658350336 00000100:00100000:2.0:1787784901.296738:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658350336 00000100:00100000:2.0:1787784901.296759:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1639 00000100:00100000:2.0:1787784901.296780:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33540:x1874627658350080:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784901.296863:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1640 00000100:00100000:1.0:1787784901.296875:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33539:x1874627658350336:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784901.296922:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33540:x1874627658350080:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 153us (671us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.296925:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98952175a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33539:x1874627658350336:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 55us (568us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.296930:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1639 00000100:00100000:1.0:1787784901.296934:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1640 00000100:00100000:0.0:1787784901.302979:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.303027:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.303072:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.303257:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658350720 00000100:00100000:1.0:1787784901.303270:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658350720 00000100:00100000:1.0:1787784901.303342:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 497 00000100:00100000:1.0:1787784901.303353:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2356:x1874627658350720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784901.303383:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.303410:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc88d4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f3a expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.303452:0:6670:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x23:0x0] initial LVB size: 158, mtime: 0x6a8f6ec4, atime: 0x0, ctime: 0x6a8f6ec4, blocks: 0x1 00010000:00010000:1.0:1787784901.303478:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1787784901.303484:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.303497:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc88d4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f3a expref: 22 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.303518:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000be20fd6d/0xb0ce63137acc88d4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f3a expref: 22 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.303534:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df200, rc 0) 00000100:00100000:2.0:1787784901.303545:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658350464 00000100:00100000:2.0:1787784901.303549:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658350464 00002000:00100000:2.0:1787784901.303588:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x20:0x0] (0->4095) req@ffff989521758780 x1874627658350464/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784901.303598:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989521758780 with x1874627658350464 ext(0->4095) 00000100:00100000:1.0:1787784901.303600:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2356:x1874627658350720:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 249us (527us total) trans 0 rc 0/0 00002000:00010000:2.0:1787784901.303604:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989521758780 00000100:00100000:1.0:1787784901.303607:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 497 00000100:00100000:2.0:1787784901.303620:0:18437:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658350592 00000100:00100000:2.0:1787784901.303622:0:18437:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658350592 00002000:00100000:2.0:1787784901.303636:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x22:0x0] (0->4095) req@ffff98953f7b0780 x1874627658350592/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787784901.303636:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 130 00000100:00100000:1.0:1787784901.303642:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2355:x1874627658350464:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00002000:00100000:2.0:1787784901.303645:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f7b0780 with x1874627658350592 ext(0->4095) 00002000:00010000:2.0:1787784901.303648:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98953f7b0780 00000100:00100000:2.0:1787784901.303661:0:18437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 131 00000100:00100000:2.0:1787784901.303667:0:18437:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2355:x1874627658350592:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1787784901.303851:0:18437:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2355:x1874627658350592:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 183us (822us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787784901.303858:0:18437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 131 00002000:00100000:2.0:1787784901.303865:0:18437:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x22:0x0] (0->4095) req@ffff98953f7b0780 x1874627658350592/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/4536 e 0 to 0 dl 1787784912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784901.303873:0:18437:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f7b0780 with x1874627658350592 ext(0->4095) 00002000:00010000:2.0:1787784901.303877:0:18437:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98953f7b0780 00000100:00100000:1.0:1787784901.304430:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521758780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2355:x1874627658350464:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 784us (1462us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787784901.304440:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 130 00002000:00100000:1.0:1787784901.304447:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x20:0x0] (0->4095) req@ffff989521758780 x1874627658350464/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/4536 e 0 to 0 dl 1787784912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.304456:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989521758780 with x1874627658350464 ext(0->4095) 00002000:00010000:1.0:1787784901.304459:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989521758780 00000100:00100000:2.0:1787784901.310279:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.310354:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.310494:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658350848 00000100:00100000:3.0:1787784901.310508:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658350848 00000100:00100000:3.0:1787784901.310580:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1641 00000100:00100000:3.0:1787784901.310592:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:33541:x1874627658350848:12345-192.168.203.43@tcp:101:bash.0 00010000:00010000:3.0:1787784901.310620:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.310648:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc88db lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f41 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310745:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310763:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310788:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310808:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310816:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310823:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310907:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310915:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310922:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310929:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.310934:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310943:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310957:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784901.310960:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.310966:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310972:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc88e2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310987:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310993:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.310999:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.311004:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.311007:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.311013:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.311021:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff9895198b5000) 00010000:00010000:3.0:1787784901.311023:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.311026:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.311032:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc88e9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.311071:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc88db lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f41 expref: 120 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.311083:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784901.311085:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.311089:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc88db lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f41 expref: 120 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.311104:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784901.311219:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:33541:x1874627658350848:12345-192.168.203.43@tcp:101:bash.0 Request processed in 629us (954us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784901.311228:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1641 00000100:00100000:1.0:1787784901.312538:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658350976 00000100:00100000:1.0:1787784901.312551:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658350976 00000100:00100000:1.0:1787784901.312618:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 90 00000100:00100000:1.0:1787784901.312635:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:2355:x1874627658350976:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784901.312679:0:5816:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.312698:0:5816:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc88f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f48 expref: -99 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.312750:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.312763:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.312789:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.312808:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.312815:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.312822:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313033:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313042:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313112:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313122:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.313129:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313137:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313208:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.313212:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787784901.313323:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.313589:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658351232 00000100:00100000:3.0:1787784901.313595:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658351232 00000100:00100000:3.0:1787784901.313638:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1642 00000100:00100000:3.0:1787784901.313647:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+96:33540:x1874627658351232:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1787784901.313738:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+96:33540:x1874627658351232:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 91us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.313746:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1642 00010000:00010000:1.0:1787784901.313917:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313932:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc88f7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313953:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.313960:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.314813:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.314820:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.314995:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.315006:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.315026:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xae1c5c33 (ffff98951b73b500) 00010000:00010000:1.0:1787784901.315030:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.315035:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.315041:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc88fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.315611:0:5816:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc88f0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f48 expref: 120 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.315625:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.315763:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784901.315769:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc88f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f48 expref: 120 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.315792:0:5816:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc -2) 00000100:00100000:1.0:1787784901.316092:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402ade580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:2355:x1874627658350976:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 3456us (5736us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784901.316103:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 90 00000100:00100000:2.0:1787784901.318920:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.319019:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658351360 00000100:00100000:0.0:1787784901.319032:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658351360 00000100:00100000:0.0:1787784901.319110:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1643 00000100:00100000:0.0:1787784901.319121:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+96:33538:x1874627658351360:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784901.319173:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784901.319201:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8905 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f4f expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.319244:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.319260:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.319273:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.319407:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8905 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 98 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.319426:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 97 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.319435:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:0.0:1787784901.319438:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784901.319442:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e800, rc 0) 00000100:00100000:0.0:1787784901.319516:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402add680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33538:x1874627658351360:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 397us (610us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.319523:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1643 00000100:00100000:0.0:1787784901.322017:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.322338:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658351488 00000100:00100000:2.0:1787784901.322349:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658351488 00000100:00100000:2.0:1787784901.322419:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1644 00000100:00100000:2.0:1787784901.322431:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+97:33539:x1874627658351488:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.322455:0:5813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.322481:0:5813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8913 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f56 expref: -99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.323312:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.323344:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.323363:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.323493:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8913 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.323521:0:5813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 98 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.323530:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784901.323534:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784901.323537:0:5813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111600, rc 0) 00000100:00100000:2.0:1787784901.323610:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+98:33539:x1874627658351488:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 1182us (1605us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.323618:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1644 00000100:00100000:1.0:1787784901.331259:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.331381:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658351616 00000100:00100000:0.0:1787784901.331399:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658351616 00000100:00100000:0.0:1787784901.331472:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1645 00000100:00100000:0.0:1787784901.331482:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+98:33539:x1874627658351616:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784901.331558:0:20438:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x34:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784901.331704:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+98:33539:x1874627658351616:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 222us (559us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.331711:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1645 00000100:00100000:0.0:1787784901.336920:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.337848:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658351744 00000100:00100000:1.0:1787784901.337865:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658351744 00002000:00100000:1.0:1787784901.337946:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x23:0x0] (0->4095) req@ffff98953f7b3840 x1874627658351744/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.337961:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f7b3840 with x1874627658351744 ext(0->4095) 00002000:00010000:1.0:1787784901.337969:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98953f7b3840 00000100:00100000:1.0:1787784901.338017:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 132 00000100:00100000:1.0:1787784901.338029:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2355:x1874627658351744:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784901.339465:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.339729:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.339757:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.339803:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658352000 00000100:00100000:3.0:1787784901.339811:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658352000 00000100:00100000:3.0:1787784901.339860:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658352128 00000100:00100000:3.0:1787784901.339863:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658352128 00000100:00100000:3.0:1787784901.339886:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1646 00000100:00100000:3.0:1787784901.339896:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+99:33538:x1874627658352000:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1787784901.339948:0:10850:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x33:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1787784901.340084:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+99:33538:x1874627658352000:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 189us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.340093:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1646 00000100:00100000:3.0:1787784901.340113:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1647 00000100:00100000:3.0:1787784901.340121:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+98:33542:x1874627658352128:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:3.0:1787784901.340131:0:10850:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.340186:0:10850:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8921 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f5d expref: -99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340255:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340273:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340286:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340303:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340311:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340317:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.340399:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2355:x1874627658351744:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 2371us (3494us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787784901.340409:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 132 00002000:00100000:1.0:1787784901.340419:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x23:0x0] (0->4095) req@ffff98953f7b3840 x1874627658351744/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/4536 e 0 to 0 dl 1787784912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.340430:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98953f7b3840 with x1874627658351744 ext(0->4095) 00002000:00010000:1.0:1787784901.340437:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff98953f7b3840 00000100:00100000:2.0:1787784901.340593:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658351872 00000100:00100000:2.0:1787784901.340599:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658351872 00000100:00100000:2.0:1787784901.340642:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 498 00000100:00100000:2.0:1787784901.340651:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2354:x1874627658351872:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1787784901.340743:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340751:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340758:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.340826:0:22706:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340829:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.340834:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340842:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.340842:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.340847:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.340852:0:22706:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340856:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784901.340859:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.340865:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340871:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8928 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340887:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340893:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340899:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340904:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.340907:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340914:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340926:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff9895198b5000) 00010000:00010000:3.0:1787784901.340929:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.340932:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340938:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc892f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.340953:0:22706:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340974:0:10850:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8921 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f5d expref: 99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.340979:0:22706:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340987:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784901.340987:0:22706:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.340989:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.340993:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8921 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f5d expref: 99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.340995:0:22706:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.341002:0:10850:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00010000:00010000:2.0:1787784901.341002:0:22706:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784901.341005:0:22706:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.341018:0:22706:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.341029:0:22706:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.341602:0:22706:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008741dc95/0xb0ce63137acc8936 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.341997:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+98:33542:x1874627658352128:12345-192.168.203.43@tcp:101:rm.0 Request processed in 922us (1284us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784901.342009:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1647 00000100:00100000:2.0:1787784901.342267:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2354:x1874627658351872:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 1005us (2200us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.342278:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 498 00000100:00100000:3.0:1787784901.345976:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.346999:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.348271:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658352384 00000100:00100000:1.0:1787784901.348288:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658352384 00000100:00100000:2.0:1787784901.348388:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658352256 00000100:00100000:2.0:1787784901.348393:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658352256 00000100:00100000:2.0:1787784901.348437:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1648 00000100:00100000:2.0:1787784901.348449:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+99:33540:x1874627658352384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.348477:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.348514:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc893d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f6b expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348559:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348575:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348594:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348739:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc893d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348760:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 100 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348785:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff989520108100) 00010000:00010000:2.0:1787784901.348789:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784901.348794:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111400, rc 0) 00000100:00100000:2.0:1787784901.348861:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+100:33540:x1874627658352384:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 414us (2899us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.348869:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1648 00000100:00100000:2.0:1787784901.348890:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1649 00000100:00100000:2.0:1787784901.348896:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539378780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+99:33542:x1874627658352256:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784901.348904:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.348913:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc894b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f64 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348951:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348961:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348974:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348988:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.348995:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349001:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349096:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349104:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349110:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349117:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.349122:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349130:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349142:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784901.349144:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.349188:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349196:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc8952 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349206:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349212:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349218:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349223:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.349226:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349232:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349237:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff989520131800) 00010000:00010000:2.0:1787784901.349239:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.349243:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349248:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8959 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349279:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc894b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f64 expref: 100 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349294:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784901.349297:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.349300:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc894b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f64 expref: 100 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.349309:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111000, rc 0) 00000100:00100000:2.0:1787784901.349336:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539378780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+99:33542:x1874627658352256:12345-192.168.203.43@tcp:101:rm.0 Request processed in 440us (2343us total) trans 0 rc 301/301 00000100:00100000:2.0:1787784901.349343:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1649 00000100:00100000:2.0:1787784901.349460:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.349568:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.353293:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658352640 00000100:00100000:3.0:1787784901.353316:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658352640 00000100:00100000:3.0:1787784901.353417:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1650 00000100:00100000:3.0:1787784901.353431:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+99:33539:x1874627658352640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784901.353556:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.353578:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.353588:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.353635:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 99 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.353854:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.354189:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658352896 00000100:00100000:0.0:1787784901.354195:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658352896 00000100:00100000:0.0:1787784901.354222:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1651 00000100:00100000:0.0:1787784901.354230:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+100:33540:x1874627658352896:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784901.354254:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784901.354380:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+100:33540:x1874627658352896:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 150us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.354389:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1651 00000100:00100000:2.0:1787784901.357139:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658352768 00000100:00100000:2.0:1787784901.357177:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658352768 00000100:00100000:2.0:1787784901.357400:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 499 00000100:00100000:2.0:1787784901.357413:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658352768:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.357474:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357508:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.357514:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.357522:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357540:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357585:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357592:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357598:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357603:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784901.357606:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357612:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357625:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.357630:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002113c530/0xb0ce63137acc8967 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.357727:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658352768:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 316us (8157us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.357738:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 499 00010000:00010000:3.0:1787784901.360957:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 99 pid: 5813 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.360990:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304ad00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668460544:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784901.361116:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784901.361219:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.365141:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304ad00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668460544:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784901.365210:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.368127:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.368316:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658353152 00000100:00100000:2.0:1787784901.368330:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658353152 00010000:00010000:2.0:1787784901.368384:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 100 pid: 5813 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.368428:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 473 00000100:00100000:2.0:1787784901.368439:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+100:2356:x1874627658353152:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784901.368458:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784901.368466:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 100 pid: 5813 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.368481:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 100 pid: 5813 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.368522:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc891a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f56 expref: 100 pid: 5813 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.368543:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784901.368549:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.368562:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.368572:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.368606:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784901.368678:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+99:2356:x1874627658353152:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 234us (904us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.368687:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 473 00010000:00010000:3.0:1787784901.370038:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370066:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370380:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370390:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370398:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370405:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed6800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.370411:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370420:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370436:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:3.0:1787784901.370440:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787784901.370445:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.370452:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8960 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.370825:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658353024 00000100:00100000:1.0:1787784901.370837:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658353024 00000100:00100000:2.0:1787784901.370851:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.370893:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 500 00000100:00100000:1.0:1787784901.370903:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2354:x1874627658353024:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784901.370954:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 3/1,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.370973:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.370978:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.370985:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.371108:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658353408 00000100:00100000:0.0:1787784901.371113:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658353408 00000100:00100000:0.0:1787784901.371175:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1652 00000100:00100000:0.0:1787784901.371183:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+100:33538:x1874627658353408:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1787784901.371254:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.371269:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.371279:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.371320:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.371335:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 100 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.371352:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668460672:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784901.371456:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f900 going to sleep for 11 seconds 00010000:00010000:1.0:1787784901.371537:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.371577:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.371585:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.371593:0:6670:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.371601:0:6670:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784901.371606:0:6670:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.371615:0:6670:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.371628:0:6670:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.371637:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc896e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.371702:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2354:x1874627658353024:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 800us (10483us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.371713:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 500 00000100:00100000:3.0:1787784901.373635:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+100:33539:x1874627658352640:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 20194us (24161us total) trans 4294968380 rc 0/0 00000100:00100000:3.0:1787784901.373653:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1650 00000100:00100000:0.0:1787784901.379573:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668460672:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784901.379635:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.379847:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.379889:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.379969:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.380001:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.380078:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658353664 00000100:00100000:2.0:1787784901.380087:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658353664 00000100:00100000:2.0:1787784901.380175:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658354176 00000100:00100000:2.0:1787784901.380179:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658354176 00000100:00100000:2.0:1787784901.380205:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1653 00000100:00100000:2.0:1787784901.380215:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+101:33540:x1874627658353664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.380311:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.380328:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.380338:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.380390:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 101 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.380403:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 101 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.380417:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adf840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668460800:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784901.380578:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658353792 00000100:00100000:1.0:1787784901.380584:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658353792 00010000:00010000:1.0:1787784901.380612:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 102 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.380640:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 474 00000100:00100000:1.0:1787784901.380648:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+102:2357:x1874627658353792:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784901.380661:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784901.380667:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 102 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.380692:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 102 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.380714:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc890c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f4f expref: 102 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.380733:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:1.0:1787784901.380739:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.380748:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.380759:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.380836:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1654 00000100:00100000:3.0:1787784901.380848:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+101:33543:x1874627658354176:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:3.0:1787784901.380863:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.380875:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8983 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f72 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.380897:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784901.380928:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+102:2357:x1874627658353792:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 279us (1038us total) trans 0 rc 0/0 00010000:00010000:3.0:1787784901.380929:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.380937:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 474 00010000:00010000:3.0:1787784901.380939:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.380947:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.380963:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.380969:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.380976:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.381038:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658354048 00000100:00100000:0.0:1787784901.381043:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658354048 00000100:00100000:0.0:1787784901.381085:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 342 00000100:00100000:0.0:1787784901.381093:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+102:33539:x1874627658354048:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00010000:00010000:3.0:1787784901.381184:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381192:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381199:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381209:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.381213:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381221:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381231:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784901.381233:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.381237:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381243:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc898a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381252:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381257:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381262:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381267:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784901.381270:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381276:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381286:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff9895198b5000) 00010000:00010000:3.0:1787784901.381288:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.381291:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381296:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8991 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381323:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8983 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f72 expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381333:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00000100:00100000:0.0:1787784901.381334:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+103:33539:x1874627658354048:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 243us (1365us total) trans 4294968381 rc 0/0 00010000:00010000:3.0:1787784901.381335:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784901.381339:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7000/0xb0ce63137acc8983 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f72 expref: 103 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.381348:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:0.0:1787784901.381358:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 342 00010000:00010000:0.0:1787784901.381391:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381399:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381561:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381568:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381573:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381578:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784901.381581:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381587:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381595:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:0.0:1787784901.381597:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784901.381600:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.381606:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8975 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.381652:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403049a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+102:33543:x1874627658354176:12345-192.168.203.43@tcp:101:cp.0 Request processed in 806us (1649us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784901.381658:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1654 00000100:00100000:0.0:1787784901.381742:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+101:33538:x1874627658353408:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 10560us (10895us total) trans 4294968382 rc 0/0 00000100:00100000:0.0:1787784901.381750:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1652 00000100:00100000:2.0:1787784901.381909:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be840 going to sleep for 11 seconds 00000100:00100000:3.0:1787784901.384233:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.384322:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.384569:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658354560 00000100:00100000:0.0:1787784901.384581:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658354560 00000100:00100000:0.0:1787784901.384647:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 343 00000100:00100000:0.0:1787784901.384662:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+101:33538:x1874627658354560:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784901.384681:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658354304 00000100:00100000:1.0:1787784901.384687:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658354304 00000100:00100000:1.0:1787784901.384720:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1655 00000100:00100000:1.0:1787784901.384728:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+102:33543:x1874627658354304:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784901.384738:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.384762:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8998 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f79 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384841:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384859:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384872:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384888:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384894:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384900:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384969:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384980:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.384986:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+104:33538:x1874627658354560:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 330us (663us total) trans 4294968383 rc 0/0 00010000:00010000:1.0:1787784901.384987:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.384994:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.384999:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385007:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385020:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.385024:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.385029:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385035:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc899f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385048:0:5811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385053:0:5811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385058:0:5811:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385063:0:5811:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.385067:0:5811:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385072:0:5811:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385078:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff98951b73b500) 00010000:00010000:1.0:1787784901.385080:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.385083:0:5811:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385088:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df800/0xb0ce63137acc89a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.385091:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668460800:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784901.385116:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8998 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f79 expref: 104 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.385120:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385127:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.385129:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.385132:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dfc00/0xb0ce63137acc8998 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f79 expref: 104 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.385141:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00000100:00100000:1.0:1787784901.385184:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a93c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+103:33543:x1874627658354304:12345-192.168.203.43@tcp:101:cp.0 Request processed in 457us (963us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784901.385190:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1655 00000100:00100000:0.0:1787784901.386309:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 343 00000100:00100000:2.0:1787784901.386795:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.386867:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658354688 00000100:00100000:3.0:1787784901.386875:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658354688 00010000:00010000:3.0:1787784901.386916:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 102 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.386962:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 475 00000100:00100000:3.0:1787784901.386972:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+102:2356:x1874627658354688:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784901.386989:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784901.386996:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 102 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.387009:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 102 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.387033:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8944 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f6b expref: 102 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.387101:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff989520108100) 00010000:00010000:3.0:1787784901.387107:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.387119:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.387127:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.387850:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784901.387918:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+101:2356:x1874627658354688:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 944us (1122us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.387927:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 475 00010000:00010000:0.0:1787784901.388030:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388041:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388634:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388645:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388652:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388659:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784901.388664:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388673:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388691:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff989520108100) 00010000:00010000:0.0:1787784901.388694:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784901.388699:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.388705:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc897c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.388900:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+101:33540:x1874627658353664:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 8684us (9053us total) trans 4294968384 rc 0/0 00000100:00100000:0.0:1787784901.388910:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1653 00000040:00080000:1.0:1787784901.389282:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784901.389288:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784901.389327:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 54 count 12 00000040:00080000:1.0:1787784901.389344:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784901.389603:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668460928:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784901.389675:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a729c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784901.391736:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.391940:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658354944 00000100:00100000:2.0:1787784901.391953:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658354944 00000100:00100000:2.0:1787784901.392024:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1656 00000100:00100000:2.0:1787784901.392036:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+102:33543:x1874627658354944:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784901.392093:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.392120:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc89ad lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f80 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.392186:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.392260:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658355200 00000100:00100000:0.0:1787784901.392265:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658355200 00000100:00100000:0.0:1787784901.392301:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 344 00010000:00010000:2.0:1787784901.392303:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.392309:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+104:33540:x1874627658355200:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.392320:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 118 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.392329:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.392479:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 118 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.392494:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 118 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.392559:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33540:x1874627658355200:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 250us (372us total) trans 4294968385 rc 0/0 00000100:00100000:0.0:1787784901.392568:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 344 00000100:00100000:2.0:1787784901.392581:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adc780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668461056:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784901.392643:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be840 going to sleep for 11 seconds 00000100:00100000:3.0:1787784901.394638:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.394726:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658355328 00000100:00100000:2.0:1787784901.394739:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658355328 00000100:00100000:1.0:1787784901.394847:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668460928:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784901.394863:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1657 00000100:00100000:2.0:1787784901.394873:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33544:x1874627658355328:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784901.394912:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x36:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784901.395031:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+104:33544:x1874627658355328:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 159us (403us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.395040:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1657 00000100:00100000:0.0:1787784901.400040:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.400395:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658355456 00000100:00100000:1.0:1787784901.400409:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658355456 00010000:00010000:1.0:1787784901.400467:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 119 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.400522:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 476 00000100:00100000:1.0:1787784901.400533:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:2356:x1874627658355456:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784901.400552:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784901.400561:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 119 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.400578:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 119 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.400608:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784901.400615:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.400628:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.400638:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.400664:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784901.400737:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:2356:x1874627658355456:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 206us (714us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.400746:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 476 00010000:00010000:2.0:1787784901.401124:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6000/0xb0ce63137acc88b1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f1e expref: 118 pid: 5811 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.401184:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adc780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668461056:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784901.401210:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.401221:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.401226:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.401253:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.401268:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.401276:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787784901.402216:0:5812:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x24:0x0], index 1: 0 00010000:00010000:2.0:1787784901.402443:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc89c2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402457:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc89c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402468:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc89c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402511:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402519:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402526:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402533:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.402538:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402547:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402565:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784901.402568:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.402573:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402579:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc89b4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784901.402592:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000962dbd1f reply state 000000006c7ec1c7 transno 4294968386 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.402601:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787784901.402644:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc89ad lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f80 expref: 105 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402658:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110200/0xb0ce63137acc89c2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f80 expref: 104 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.402667:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff989503a6d200) 00010000:00010000:2.0:1787784901.402789:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784901.402795:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110200, rc 0) 00000100:00100000:2.0:1787784901.402885:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33543:x1874627658354944:12345-192.168.203.43@tcp:101:cp.0 Request processed in 10850us (11159us total) trans 4294968386 rc 0/0 00000100:00100000:2.0:1787784901.402895:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1656 00000100:00100000:2.0:1787784901.404302:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.404387:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658355712 00000100:00100000:0.0:1787784901.404399:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658355712 00000100:00100000:0.0:1787784901.404463:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1658 00000100:00100000:0.0:1787784901.404475:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33544:x1874627658355712:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784901.404505:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784901.404528:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc89d0 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f87 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.404647:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc89d7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.404662:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc89d7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.404688:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc89d7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787784901.404700:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x36:0x0]: returning size 158 00010000:00010000:1.0:1787784901.406265:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.406282:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.406290:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.406297:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784901.406302:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.406311:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.406329:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff989520131800) 00010000:00010000:1.0:1787784901.406333:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784901.406611:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.406621:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111200/0xb0ce63137acc89bb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.407053:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc89d0 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f87 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.407073:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc89d7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f87 expref: 106 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.407080:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff989520108e00) 00010000:00010000:0.0:1787784901.407083:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787784901.407086:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704fc00, rc 0) 00000100:00100000:0.0:1787784901.407174:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+106:33544:x1874627658355712:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 2700us (2880us total) trans 4294968387 rc 0/0 00000100:00100000:0.0:1787784901.407184:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1658 00000100:00100000:2.0:1787784901.409272:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.409599:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658355840 00000100:00100000:1.0:1787784901.409612:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658355840 00000100:00100000:2.0:1787784901.409738:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00100000:2.0:1787784901.409751:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658355840:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:2.0:1787784901.409788:0:22706:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.409817:0:22706:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f8e expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784901.409849:0:22706:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x24:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1787784901.409873:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1787784901.409878:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.409890:0:22706:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 23 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.409909:0:22706:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 23 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.409924:0:22706:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111400, rc 0) 00000100:00100000:2.0:1787784901.410127:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2357:x1874627658355840:12345-192.168.203.43@tcp:101:cp.0 Request processed in 378us (867us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.410136:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00100000:0.0:1787784901.414327:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.414495:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.414600:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658356096 00000100:00100000:1.0:1787784901.414613:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658356096 00000100:00100000:1.0:1787784901.414714:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 76 00000100:00100000:1.0:1787784901.414726:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+106:2355:x1874627658356096:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784901.414748:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784901.414793:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc89d7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f87 expref: 106 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.414818:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff989520108e00) 00010000:00010000:1.0:1787784901.414824:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784901.414891:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+105:2355:x1874627658356096:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 168us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.414899:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 76 00000100:00100000:2.0:1787784901.419894:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658355968 00000100:00100000:2.0:1787784901.419916:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658355968 00000100:00100000:2.0:1787784901.420040:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 502 00000100:00100000:2.0:1787784901.420054:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+21:2356:x1874627658355968:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1787784901.420117:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.420142:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 21 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.420183:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784901.420189:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787784901.420194:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 4/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.420242:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 21 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.420259:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 21 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.420276:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668461184:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784901.420390:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be840 going to sleep for 11 seconds 00000100:00100000:0.0:1787784901.423906:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.423997:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658356224 00000100:00100000:1.0:1787784901.424010:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658356224 00000100:00100000:1.0:1787784901.424081:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 503 00000100:00100000:1.0:1787784901.424093:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+23:33543:x1874627658356224:12345-192.168.203.43@tcp:101:cp.0 00010000:00010000:1.0:1787784901.424118:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.424142:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f95 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.424230:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 24 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.424239:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.424244:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.424249:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 24 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.424296:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 24 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.424311:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 24 pid: 22706 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.424328:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668461312:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784901.424430:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a720c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784901.426315:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.426516:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adda40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668461184:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784901.426548:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.426645:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658356352 00000100:00100000:1.0:1787784901.426655:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658356352 00002000:00100000:1.0:1787784901.426720:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x21:0x0] (0->157) req@ffff989402adf840 x1874627658356352/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.426731:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402adf840 with x1874627658356352 ext(0->157) 00010000:00010000:1.0:1787784901.426740:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 22 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.426754:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 22 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.426761:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989402adf840 00000100:00100000:1.0:1787784901.426811:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 25 00000100:00100000:1.0:1787784901.426823:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658356352:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:1.0:1787784901.428533:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2357:x1874627658356352:12345-192.168.203.43@tcp:4:cp.0 Request processed in 1701us (2219us total) trans 4294967380 rc 0/0 00000100:00100000:1.0:1787784901.428548:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 25 00002000:00100000:1.0:1787784901.428562:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x21:0x0] (0->157) req@ffff989402adf840 x1874627658356352/t4294967380(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 648/448 e 0 to 0 dl 1787784912 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.428572:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989402adf840 with x1874627658356352 ext(0->157) 00010000:00010000:1.0:1787784901.428581:0:6675:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 24 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.428596:0:6675:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 24 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.428603:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff989402adf840 00000100:00100000:1.0:1787784901.428666:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668461312:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784901.428708:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 24 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.428730:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393dfc00, rc 0) 00000100:00100000:1.0:1787784901.428754:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:33543:x1874627658356224:12345-192.168.203.43@tcp:101:cp.0 Request processed in 4664us (4863us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.428761:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 503 00000100:00100000:2.0:1787784901.430069:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.430199:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658356480 00000100:00100000:0.0:1787784901.430208:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658356480 00010000:00010000:0.0:1787784901.430250:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 24 pid: 22706 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.430290:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 477 00000100:00100000:0.0:1787784901.430301:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2357:x1874627658356480:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784901.430318:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784901.430346:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 24 pid: 22706 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.430370:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 24 pid: 22706 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.430398:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d0f5b066/0xb0ce63137acc89e5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f8e expref: 24 pid: 22706 timeout: 792 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.430418:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784901.430422:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784901.430431:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 23 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.430443:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 23 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.430494:0:5797:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 23 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.430513:0:5797:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668461440:192.168.203.43@tcp:105: 00000100:00100000:0.0:1787784901.430741:0:5797:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f900 going to sleep for 11 seconds 00000100:00100000:2.0:1787784901.436831:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.437061:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658356608 00000100:00100000:1.0:1787784901.437073:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658356608 00010000:00010000:1.0:1787784901.437127:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 24 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.440614:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 478 00000100:00100000:1.0:1787784901.440641:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2355:x1874627658356608:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784901.440681:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784901.440718:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 24 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.440742:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 24 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.440787:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000113dd65f/0xb0ce63137acc8809 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1eb5 expref: 24 pid: 10688 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.440807:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784901.440812:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784901.440819:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.440831:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.440867:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784901.440931:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2355:x1874627658356608:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 301us (4114us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.440940:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 478 00010000:00010000:1.0:1787784901.440982:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.440997:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.441027:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.441038:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.441046:0:13825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.441054:0:13825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111000 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784901.441059:0:13825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.441068:0:13825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.441080:0:13825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.441088:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f9e1d77/0xb0ce63137acc89ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.441530:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2356:x1874627658355968:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 21096us (26651us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.441542:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 502 00000100:00100000:2.0:1787784901.444020:0:5797:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b2580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5797:1874627668461440:192.168.203.43@tcp:105: 00010000:00010000:2.0:1787784901.444071:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784901.444134:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2357:x1874627658356480:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 13833us (14070us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.444176:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 477 00000100:00100000:2.0:1787784901.447119:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.447245:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.447388:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658357120 00000100:00100000:1.0:1787784901.447401:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658357120 00000100:00100000:1.0:1787784901.447483:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1659 00000100:00100000:1.0:1787784901.447496:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33544:x1874627658357120:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784901.447684:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33544:x1874627658357120:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 179us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.447693:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1659 00000100:00100000:3.0:1787784901.451806:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.451961:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658357248 00000100:00100000:1.0:1787784901.451974:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658357248 00000100:00100000:1.0:1787784901.452036:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 504 00000100:00100000:1.0:1787784901.452048:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2357:x1874627658357248:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1787784901.452071:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.452102:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf95fe06/0xb0ce63137acc89fa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1f9c expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787784901.452136:0:10688:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x21:0x0] initial LVB size: 158, mtime: 0x6a8f6ec4, atime: 0x0, ctime: 0x6a8f6ec4, blocks: 0x1 00010000:00010000:1.0:1787784901.452198:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1787784901.452204:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.452215:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf95fe06/0xb0ce63137acc89fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f9c expref: 23 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.452241:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bf95fe06/0xb0ce63137acc89fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f9c expref: 23 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.452256:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df800, rc 0) 00000100:00100000:1.0:1787784901.452324:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2357:x1874627658357248:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 278us (533us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.452331:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 504 00000100:00100000:3.0:1787784901.455198:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.455341:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658357504 00000100:00100000:2.0:1787784901.455363:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658357504 00000100:00100000:2.0:1787784901.455447:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1660 00000100:00100000:2.0:1787784901.455458:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33544:x1874627658357504:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1787784901.455615:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f083c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33544:x1874627658357504:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 158us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.455623:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1660 00000100:00100000:0.0:1787784901.457700:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658356864 00000100:00100000:0.0:1787784901.457716:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658356864 00000100:00100000:1.0:1787784901.457908:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 345 00000100:00100000:1.0:1787784901.457920:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+105:33543:x1874627658356864:12345-192.168.203.43@tcp:35:cp.0 00000100:00100000:1.0:1787784901.458410:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+106:33543:x1874627658356864:12345-192.168.203.43@tcp:35:cp.0 Request processed in 490us (11304us total) trans 4294968388 rc 0/0 00000100:00100000:1.0:1787784901.458418:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 345 00000100:00100000:3.0:1787784901.478366:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.478446:0:6676:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658357632 00000100:00100000:1.0:1787784901.478461:0:6676:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658357632 00002000:00100000:1.0:1787784901.478554:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x21:0x0] (0->4095) req@ffff989518f08b40 x1874627658357632/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/0 e 0 to 0 dl 1787784912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.478567:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f08b40 with x1874627658357632 ext(0->4095) 00002000:00010000:1.0:1787784901.478574:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989518f08b40 00000100:00100000:1.0:1787784901.478628:0:6676:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 133 00000100:00100000:1.0:1787784901.478638:0:6676:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2356:x1874627658357632:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1787784901.478932:0:6676:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f08b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2356:x1874627658357632:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 293us (589us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787784901.478944:0:6676:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 133 00002000:00100000:1.0:1787784901.478951:0:6676:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x21:0x0] (0->4095) req@ffff989518f08b40 x1874627658357632/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:27/0 lens 488/4536 e 0 to 0 dl 1787784912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787784901.478960:0:6676:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989518f08b40 with x1874627658357632 ext(0->4095) 00002000:00010000:1.0:1787784901.478963:0:6676:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff989518f08b40 00000100:00100000:2.0:1787784901.496799:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.497561:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658357760 00000100:00100000:0.0:1787784901.497579:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658357760 00000100:00100000:0.0:1787784901.497684:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1661 00000100:00100000:0.0:1787784901.497699:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+106:33544:x1874627658357760:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784901.497779:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784901.497817:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8a01 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fa3 expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.497879:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.497897:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.497913:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.498060:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8a01 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 108 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.498089:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.498099:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff989520108e00) 00010000:00010000:0.0:1787784901.498103:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784901.498106:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e000, rc 0) 00000100:00100000:0.0:1787784901.498242:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+107:33544:x1874627658357760:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 546us (1458us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.498250:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1661 00000100:00100000:3.0:1787784901.528026:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.528251:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658357888 00000100:00100000:2.0:1787784901.528264:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658357888 00000100:00100000:2.0:1787784901.528349:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1662 00000100:00100000:2.0:1787784901.528360:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+107:33544:x1874627658357888:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784901.528453:0:5811:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x36:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784901.528619:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989518f0b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+107:33544:x1874627658357888:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 260us (608us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.528628:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1662 00000100:00100000:3.0:1787784901.538336:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.539996:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658358016 00000100:00100000:1.0:1787784901.540020:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658358016 00000100:00100000:1.0:1787784901.540124:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 505 00000100:00100000:1.0:1787784901.540136:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2357:x1874627658358016:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784901.540241:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540265:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.540270:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784901.540278:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540295:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540336:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540344:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540359:0:6671:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540366:0:6671:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784901.540371:0:6671:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540381:0:6671:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540399:0:6671:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.540407:0:6671:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c47f952/0xb0ce63137acc8a0f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.540541:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2357:x1874627658358016:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 406us (2218us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.540550:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 505 00000100:00100000:0.0:1787784901.545001:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.545187:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658358272 00000100:00100000:3.0:1787784901.545200:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658358272 00000100:00100000:3.0:1787784901.545278:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1663 00000100:00100000:3.0:1787784901.545291:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+107:33544:x1874627658358272:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1787784901.545410:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.545589:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.545602:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.545950:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 107 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.545968:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 107 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.545986:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400a940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668461568:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784901.546105:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895033be300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784901.551327:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784901.551503:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658358400 00000100:00100000:2.0:1787784901.551516:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658358400 00000100:00100000:2.0:1787784901.551729:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1664 00000100:00100000:2.0:1787784901.551744:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33547:x1874627658358400:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:2.0:1787784901.551785:0:20438:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784901.551815:0:20438:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8a1d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1faa expref: -99 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.551908:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.551927:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.551942:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.551959:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.551967:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.551975:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552104:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8a32 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552114:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8a32 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552122:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8a32 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784901.552876:0:20438:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x37:0x0]: returning size 158 00000004:00200000:2.0:1787784901.552884:0:20438:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1787784901.552915:0:20438:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8a32 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552934:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8a32 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552945:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552953:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552961:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552968:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.552973:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552983:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.552998:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784901.553002:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.553007:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553015:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8a24 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553036:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553044:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553050:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553055:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.553058:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553064:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553072:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff989519d44700) 00010000:00010000:2.0:1787784901.553075:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784901.553078:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553085:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111000/0xb0ce63137acc8a2b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553142:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8a1d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1faa expref: 120 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553804:0:20438:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111400/0xb0ce63137acc8a32 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1faa expref: 119 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.553821:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff989503a6d200) 00010000:00010000:2.0:1787784901.553823:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784901.553826:0:20438:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111400, rc 0) 00010000:00010000:0.0:1787784901.554002:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784901.554003:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:33547:x1874627658358400:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 2261us (2692us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784901.554014:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1664 00000100:00100000:3.0:1787784901.564016:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.565367:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658358528 00000100:00100000:1.0:1787784901.565390:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658358528 00000100:00100000:3.0:1787784901.565581:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 506 00000100:00100000:3.0:1787784901.565601:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658358528:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784901.565619:0:13825:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.565746:0:13825:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8a39 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fb1 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784901.565991:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668461696:192.168.203.43@tcp:106: 00000100:00100000:3.0:1787784901.566090:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff98952a169cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787784901.586841:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400a940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668461568:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784901.586883:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787784901.589424:0:13825:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x24:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1787784901.589445:0:13825:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x24:0x0] updating lvb mtime: 0 -> 1787784900 00002000:00010000:3.0:1787784901.589448:0:13825:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x24:0x0] updating lvb atime: 0 -> 1787784900 00002000:00010000:3.0:1787784901.589450:0:13825:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x24:0x0] updating lvb ctime: 0 -> 1787784900 00000100:00100000:3.0:1787784901.589483:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668461696:192.168.203.43@tcp:106: 00010000:00010000:3.0:1787784901.589533:0:13825:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8a39 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fb1 expref: 37 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.589562:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8a39 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fb1 expref: 37 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.589579:0:13825:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:3.0:1787784901.589672:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2356:x1874627658358528:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 24078us (25664us total) trans 0 rc 301/301 00000100:00100000:3.0:1787784901.589685:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 506 00000100:00100000:2.0:1787784901.589959:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784901.590202:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658358656 00000100:00100000:0.0:1787784901.590211:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658358656 00010000:00010000:0.0:1787784901.590255:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 108 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784901.590289:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 479 00000100:00100000:0.0:1787784901.590300:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+108:2357:x1874627658358656:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784901.590319:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784901.590327:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 108 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.590340:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 108 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.590360:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8a08 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fa3 expref: 108 pid: 20438 timeout: 792 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.590376:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff989520108e00) 00010000:00010000:0.0:1787784901.590383:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.590395:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.590404:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784901.590459:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784901.590498:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+107:2357:x1874627658358656:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 200us (539us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784901.590506:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 479 00010000:00010000:2.0:1787784901.591367:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591382:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591732:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591741:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591747:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591753:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989404ed7e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784901.591758:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591831:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591849:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff989520108e00) 00010000:00010000:2.0:1787784901.591852:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784901.591856:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784901.591862:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7e00/0xb0ce63137acc8a16 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784901.592708:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+107:33544:x1874627658358272:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 47411us (47713us total) trans 4294968389 rc 0/0 00000100:00100000:2.0:1787784901.592724:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1663 00000100:00100000:0.0:1787784901.599687:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784901.600479:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658358784 00000100:00100000:3.0:1787784901.600495:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658358784 00000100:00100000:3.0:1787784901.600576:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1665 00000100:00100000:3.0:1787784901.600588:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+119:33547:x1874627658358784:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:3.0:1787784901.600615:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784901.600644:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8a40 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fb8 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.600746:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8a47 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.600763:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8a47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.601128:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8a47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787784901.601925:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 512 00000004:00200000:3.0:1787784901.601999:0:5812:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 10 stripe count 1 MDT count 1 00000004:00010000:3.0:1787784901.602037:0:5812:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8a47 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.604502:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed7800/0xb0ce63137acc8a40 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fb8 expref: 121 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.604553:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8a47 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fb8 expref: 120 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784901.604564:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784901.604567:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787784901.604572:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6800, rc 0) 00000040:00080000:1.0:1787784901.605438:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784901.605448:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784901.605962:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 12 00000040:00080000:1.0:1787784901.605976:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784901.615467:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+120:33547:x1874627658358784:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 14860us (15773us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784901.615934:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1665 00000100:00100000:2.0:1787784901.623283:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.623366:0:5815:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658359040 00000100:00100000:1.0:1787784901.623381:0:5815:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658359040 00000100:00100000:1.0:1787784901.623462:0:5815:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 346 00000100:00100000:1.0:1787784901.623475:0:5815:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895075d3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+107:33544:x1874627658359040:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1787784901.630487:0:5815:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895075d3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+108:33544:x1874627658359040:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 6997us (7203us total) trans 4294968390 rc 0/0 00000100:00100000:1.0:1787784901.630508:0:5815:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 346 00000100:00100000:0.0:1787784901.658012:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.658413:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658359168 00000100:00100000:1.0:1787784901.658426:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658359168 00000100:00100000:1.0:1787784901.658503:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 507 00000100:00100000:1.0:1787784901.658514:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658359168:12345-192.168.203.43@tcp:101:lfs.0 00010000:00010000:1.0:1787784901.658539:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784901.658562:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c47f952/0xb0ce63137acc8a4e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fbf expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784901.658655:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668461824:192.168.203.43@tcp:106: 00000100:00100000:1.0:1787784901.658812:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a72180 going to sleep for 11 seconds 00000100:00100000:1.0:1787784901.679011:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10688:1874627668461824:192.168.203.43@tcp:106: 00010000:00010000:1.0:1787784901.679069:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c47f952/0xb0ce63137acc8a4e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fbf expref: 37 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.679102:0:10688:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c47f952/0xb0ce63137acc8a4e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fbf expref: 37 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784901.679117:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393df600, rc 0) 00000100:00100000:1.0:1787784901.679259:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658359168:12345-192.168.203.43@tcp:101:lfs.0 Request processed in 20745us (21263us total) trans 0 rc 301/301 00000100:00100000:1.0:1787784901.679270:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 507 00000100:00100000:2.0:1787784901.692669:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784901.692847:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658359424 00000100:00100000:1.0:1787784901.692859:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658359424 00000100:00100000:1.0:1787784901.692940:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1666 00000100:00100000:1.0:1787784901.692952:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+120:33547:x1874627658359424:12345-192.168.203.43@tcp:58:lfs.0 00000040:00080000:1.0:1787784901.693331:0:5811:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].54, 136 off15536 00000100:00100000:1.0:1787784901.693439:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+120:33547:x1874627658359424:12345-192.168.203.43@tcp:58:lfs.0 Request processed in 488us (785us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784901.693448:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1666 00000100:00100000:2.0:1787784902.130729:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668461952:0@lo:400:kworker.0 00000100:00100000:3.0:1787784902.130791:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668462080:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784902.131890:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784902.131982:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668462592:0@lo:400:kworker.0 00000100:00100000:3.0:1787784902.132022:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784902.132058:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1787784902.132079:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668462208:0@lo:400:kworker.0 00000100:00100000:3.0:1787784902.132105:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784902.132128:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668462080 00000100:00100000:1.0:1787784902.132142:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668462080 00000100:00100000:2.0:1787784902.132222:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784902.132239:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1787784902.132253:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668462080:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784902.132265:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404008f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668462464:0@lo:400:kworker.0 00002000:00000024:1.0:1787784902.132286:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 105447424 pending 6815744 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784902.132303:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784902.132333:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784902.132342:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404009a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668462336:0@lo:400:kworker.0 00000100:00080000:2.0:1787784902.132383:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784902.132399:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668461952 00000100:00100000:2.0:1787784902.132403:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668461952 00000100:00100000:1.0:1787784902.132411:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668462080:12345-0@lo:13:osp-pre-1-0.0 Request processed in 100us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784902.132422:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1787784902.132453:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 618 00000100:00100000:2.0:1787784902.132462:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668461952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784902.132510:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668461952:12345-0@lo:400:kworker.0 Request processed in 50us (289us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.132519:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 618 00000100:00100000:3.0:1787784902.132556:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668462720:0@lo:400:kworker.0 00000100:00100000:2.0:1787784902.132564:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668462464 00000100:00100000:2.0:1787784902.132570:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668462464 00000100:00100000:3.0:1787784902.132588:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784902.132615:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668462720 00000100:00100000:2.0:1787784902.132620:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668462720 00000100:00100000:3.0:1787784902.132647:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668462080:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784902.132649:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1787784902.132662:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668462464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784902.132704:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668462464:12345-0@lo:400:kworker.0 Request processed in 44us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.132710:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1787784902.132723:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1787784902.132729:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668462720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784902.132900:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668462592 00000100:00100000:3.0:1787784902.132906:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668462592 00000100:00100000:3.0:1787784902.133961:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668462208 00000100:00100000:2.0:1787784902.133973:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668462720:12345-0@lo:400:kworker.0 Request processed in 1234us (1376us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.133979:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668462208 00000100:00100000:2.0:1787784902.134001:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00080000:2.0:1787784902.134052:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784902.134059:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784902.134074:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784902.136050:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668461952:0@lo:400:kworker.0 00000100:00100000:2.0:1787784902.136079:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404008f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668462464:0@lo:400:kworker.0 00000100:00100000:2.0:1787784902.136144:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668462336 00000100:00100000:2.0:1787784902.136168:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668462336 00000100:00100000:2.0:1787784902.136234:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:2.0:1787784902.136244:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668462592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784902.136334:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530517840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668462592:12345-0@lo:400:kworker.0 Request processed in 91us (4311us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.136342:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:2.0:1787784902.136364:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 500 00000100:00100000:2.0:1787784902.136372:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668462208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784902.136404:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668462208:12345-0@lo:400:kworker.0 Request processed in 33us (4298us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.136410:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 500 00000100:00100000:0.0:1787784902.137804:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 501 00000100:00100000:0.0:1787784902.137829:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668462336:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784902.137928:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668462336:12345-0@lo:400:kworker.0 Request processed in 110us (3852us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784902.137936:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 501 00000100:00100000:3.0:1787784902.138294:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668462208:0@lo:400:kworker.0 00000100:00100000:3.0:1787784902.138312:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668462720:0@lo:400:kworker.0 00000100:00100000:3.0:1787784902.138325:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668462592:0@lo:400:kworker.0 00000100:00080000:2.0:1787784902.139617:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784902.139677:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404009a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668462336:0@lo:400:kworker.0 00010000:00010000:3.0:1787784902.577911:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784902.641482:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784902.641494:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784902.641714:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:1.0:1787784902.641722:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784902.642223:0:11668:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668462848:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1787784902.642344:0:11668:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506643300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784902.685714:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784902.685801:0:11668:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11668:1874627668462848:192.168.203.43@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1787784902.690251:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658359552 00000100:00100000:2.0:1787784902.690272:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658359552 00000100:00100000:2.0:1787784902.690397:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1667 00000100:00100000:2.0:1787784902.690409:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+108:33557:x1874627658359552:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1787784902.690460:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x37:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1787784902.690636:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+108:33557:x1874627658359552:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 228us (4935us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.690645:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1667 00000100:00080000:2.0:1787784902.690674:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784902.700933:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.701202:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658359808 00000100:00100000:2.0:1787784902.701216:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658359808 00000100:00100000:2.0:1787784902.701282:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1668 00000100:00100000:2.0:1787784902.701292:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+108:33557:x1874627658359808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784902.701321:0:5812:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784902.701348:0:5812:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc8a55 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fc6 expref: -99 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.701499:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc8a5c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.701517:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc8a5c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.701532:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc8a5c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787784902.701545:0:5812:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x37:0x0]: returning size 158 00010000:00010000:2.0:1787784902.701735:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111600/0xb0ce63137acc8a55 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fc6 expref: 111 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.701758:0:5812:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc8a5c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fc6 expref: 110 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.701781:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff989503a6d200) 00010000:00010000:2.0:1787784902.701785:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787784902.701789:0:5812:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110800, rc 0) 00000100:00100000:2.0:1787784902.701867:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954073a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+110:33557:x1874627658359808:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 575us (947us total) trans 4294968391 rc 0/0 00000100:00100000:2.0:1787784902.701877:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1668 00000100:00100000:2.0:1787784902.724002:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.724059:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.724101:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658360064 00000100:00100000:2.0:1787784902.724113:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658360064 00000100:00100000:2.0:1787784902.724211:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 77 00000100:00100000:2.0:1787784902.724225:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+110:2356:x1874627658360064:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784902.724248:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784902.724282:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110800/0xb0ce63137acc8a5c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fc6 expref: 110 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.724307:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff989503a6d200) 00010000:00010000:2.0:1787784902.724313:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784902.724397:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+109:2356:x1874627658360064:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 175us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.724406:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 77 00000100:00100000:1.0:1787784902.726920:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658359936 00000100:00100000:1.0:1787784902.726941:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658359936 00000100:00100000:1.0:1787784902.727050:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 508 00000100:00100000:1.0:1787784902.727063:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2356:x1874627658359936:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1787784902.727122:0:22706:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.727175:0:22706:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 23 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.727187:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x24:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1787784902.727193:0:22706:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784902.727198:0:22706:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 4/1,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.727245:0:22706:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 23 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.727262:0:22706:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 23 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784902.727280:0:22706:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bc3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668462976:192.168.203.43@tcp:104: 00000100:00080000:2.0:1787784902.727618:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784902.728544:0:22706:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066436c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784902.747112:0:22706:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bc3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:22706:1874627668462976:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784902.747174:0:22706:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784902.754448:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.754843:0:6674:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658360192 00000100:00100000:2.0:1787784902.754857:0:6674:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658360192 00002000:00100000:2.0:1787784902.754936:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x24:0x0] (0->157) req@ffff98951b5bc3c0 x1874627658360192/t0(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:28/0 lens 648/0 e 0 to 0 dl 1787784913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784902.754948:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5bc3c0 with x1874627658360192 ext(0->157) 00010000:00010000:2.0:1787784902.754960:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 24 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.754980:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 24 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784902.754988:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff98951b5bc3c0 00000100:00100000:2.0:1787784902.755018:0:6674:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 26 00000100:00100000:2.0:1787784902.755029:0:6674:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2357:x1874627658360192:12345-192.168.203.43@tcp:4:cp.0 00000100:00100000:2.0:1787784902.766467:0:6674:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2357:x1874627658360192:12345-192.168.203.43@tcp:4:cp.0 Request processed in 11420us (12015us total) trans 4294967395 rc 0/0 00000100:00100000:2.0:1787784902.766489:0:6674:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 26 00002000:00100000:2.0:1787784902.766508:0:6674:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x24:0x0] (0->157) req@ffff98951b5bc3c0 x1874627658360192/t4294967395(0) o4->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:28/0 lens 648/448 e 0 to 0 dl 1787784913 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787784902.766520:0:6674:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff98951b5bc3c0 with x1874627658360192 ext(0->157) 00010000:00010000:2.0:1787784902.766533:0:6674:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 26 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.766559:0:6674:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 26 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784902.766580:0:6674:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff98951b5bc3c0 00000100:00100000:3.0:1787784902.773917:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784902.773990:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658360320 00000100:00100000:0.0:1787784902.774004:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658360320 00010000:00010000:0.0:1787784902.774056:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 26 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784902.774119:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 480 00000100:00100000:0.0:1787784902.774132:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+26:2356:x1874627658360320:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784902.774184:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784902.774718:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 26 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.774741:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x24:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 26 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.774781:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc89f3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x24:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f95 expref: 26 pid: 6670 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.774802:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784902.774807:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787784902.774815:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.774827:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.774893:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784902.774962:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2356:x1874627658360320:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 831us (1055us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784902.774974:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 480 00010000:00010000:0.0:1787784902.775308:0:22706:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775321:0:22706:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775375:0:22706:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775390:0:22706:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775399:0:22706:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775408:0:22706:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784902.775414:0:22706:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775425:0:22706:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775438:0:22706:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.775452:0:22706:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000081a0022d/0xb0ce63137acc8a6a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 22706 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784902.775542:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2356:x1874627658359936:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 48481us (51555us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784902.775552:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 508 00000100:00100000:3.0:1787784902.777976:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784902.780127:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658360576 00000100:00100000:1.0:1787784902.780167:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658360576 00000100:00100000:1.0:1787784902.780283:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1669 00000100:00100000:1.0:1787784902.780296:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+109:33557:x1874627658360576:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784902.780468:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+109:33557:x1874627658360576:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 174us (2506us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784902.780480:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1669 00000100:00100000:1.0:1787784902.786488:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.787212:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658360704 00000100:00100000:2.0:1787784902.787227:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658360704 00000100:00100000:2.0:1787784902.787305:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00100000:2.0:1787784902.787319:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+24:2357:x1874627658360704:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1787784902.787345:0:6671:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784902.787377:0:6671:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fbd297d6/0xb0ce63137acc8a71 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fcd expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787784902.787412:0:6671:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x24:0x0] initial LVB size: 158, mtime: 0x6a8f6ec4, atime: 0x0, ctime: 0x6a8f6ec4, blocks: 0x1 00010000:00010000:2.0:1787784902.787439:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1787784902.787444:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784902.787457:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fbd297d6/0xb0ce63137acc8a71 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fcd expref: 25 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.787484:0:6671:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fbd297d6/0xb0ce63137acc8a71 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fcd expref: 25 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.787500:0:6671:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537110800, rc 0) 00000100:00100000:2.0:1787784902.787803:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2357:x1874627658360704:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 485us (1330us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.787813:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00100000:1.0:1787784902.800128:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784902.800212:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658360960 00000100:00100000:1.0:1787784902.800224:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658360960 00000100:00100000:1.0:1787784902.800294:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1670 00000100:00100000:1.0:1787784902.800305:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+109:33557:x1874627658360960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1787784902.800450:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bfc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+109:33557:x1874627658360960:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 146us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784902.800459:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1670 00000100:00100000:2.0:1787784902.807989:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784902.808270:0:6675:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658361088 00000100:00100000:3.0:1787784902.808283:0:6675:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658361088 00002000:00100000:3.0:1787784902.808353:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x24:0x0] (0->4095) req@ffff989404008b40 x1874627658361088/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:28/0 lens 488/0 e 0 to 0 dl 1787784913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784902.808373:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404008b40 with x1874627658361088 ext(0->4095) 00002000:00010000:3.0:1787784902.808380:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989404008b40 00000100:00100000:3.0:1787784902.808420:0:6675:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 134 00000100:00100000:3.0:1787784902.808432:0:6675:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2354:x1874627658361088:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1787784902.809879:0:6675:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2354:x1874627658361088:12345-192.168.203.43@tcp:3:lhsmtool_posix.0 Request processed in 1438us (1894us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787784902.809892:0:6675:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 134 00002000:00100000:3.0:1787784902.809902:0:6675:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x24:0x0] (0->4095) req@ffff989404008b40 x1874627658361088/t0(0) o3->5882e86d-aef6-465c-ae01-bd8a61753eef@192.168.203.43@tcp:28/0 lens 488/4536 e 0 to 0 dl 1787784913 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787784902.809913:0:6675:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff989404008b40 with x1874627658361088 ext(0->4095) 00002000:00010000:3.0:1787784902.809919:0:6675:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff989404008b40 00000100:00100000:1.0:1787784902.816297:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784902.816517:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658361216 00000100:00100000:0.0:1787784902.816530:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658361216 00000100:00100000:0.0:1787784902.816613:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1671 00000100:00100000:0.0:1787784902.816626:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+109:33557:x1874627658361216:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1787784902.816658:0:5811:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784902.816684:0:5811:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8a78 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fd4 expref: -99 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.816726:0:5811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.816744:0:5811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.816757:0:5811:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.817203:0:5811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8a78 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 111 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.817221:0:5811:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 110 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.817232:0:5811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff989503a6d200) 00010000:00010000:0.0:1787784902.817235:0:5811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787784902.817239:0:5811:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e800, rc 0) 00000100:00100000:0.0:1787784902.817325:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+110:33557:x1874627658361216:12345-192.168.203.43@tcp:101:lhsmtool_posix.0 Request processed in 701us (1042us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784902.817334:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1671 00000100:00100000:0.0:1787784902.826349:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784902.826504:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658361344 00000100:00100000:0.0:1787784902.828068:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658361344 00000100:00100000:0.0:1787784902.828186:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1672 00000100:00100000:0.0:1787784902.828199:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+110:33557:x1874627658361344:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1787784902.828243:0:5813:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x37:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1787784902.828393:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+110:33557:x1874627658361344:12345-192.168.203.43@tcp:53:lhsmtool_posix.0 Request processed in 196us (2057us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784902.828403:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1672 00000100:00100000:0.0:1787784902.835906:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784902.835959:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658361472 00000100:00100000:0.0:1787784902.835973:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658361472 00000100:00100000:0.0:1787784902.836044:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 510 00000100:00100000:0.0:1787784902.836056:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2357:x1874627658361472:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1787784902.836127:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836178:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784902.836185:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784902.836192:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836209:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836248:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836258:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836266:0:6669:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836275:0:6669:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fc00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:0.0:1787784902.836279:0:6669:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836289:0:6669:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836309:0:6669:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.836318:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000db4baa8e/0xb0ce63137acc8a86 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784902.836418:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+25:2357:x1874627658361472:12345-192.168.203.43@tcp:1:lhsmtool_posix.0 Request processed in 362us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784902.836427:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 510 00000040:00080000:2.0:1787784902.837668:0:22745:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784902.837680:0:22745:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784902.837757:0:22745:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:2.0:1787784902.837778:0:22745:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784902.843649:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784902.843959:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658361728 00000100:00100000:1.0:1787784902.843973:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658361728 00000100:00100000:1.0:1787784902.844085:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1673 00000100:00100000:1.0:1787784902.844098:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+110:33557:x1874627658361728:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1787784902.844226:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.844248:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 110 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.844257:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 4/0,1 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.844299:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 110 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.844320:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 110 pid: 5811 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784902.844337:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668463104:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784902.844661:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895066436c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784902.850093:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784902.850190:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658361856 00000100:00100000:3.0:1787784902.850202:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658361856 00010000:00010000:3.0:1787784902.850253:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 111 pid: 5811 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784902.850297:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 481 00000100:00100000:3.0:1787784902.850307:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+111:2357:x1874627658361856:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784902.850327:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784902.850335:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 111 pid: 5811 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.850421:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 111 pid: 5811 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.850657:0:5795:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff989503a6d200) 00010000:00010000:3.0:1787784902.850666:0:5795:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.850689:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.850698:0:5795:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.850720:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784902.851185:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e800/0xb0ce63137acc8a7f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fd4 expref: 111 pid: 5811 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784902.851205:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668463104:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784902.851238:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.851248:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.851254:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.852141:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.852278:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.852286:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.852292:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784902.852296:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.852303:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.852318:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff989503a6d200) 00010000:00010000:2.0:1787784902.852320:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1787784902.852323:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.852328:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df600/0xb0ce63137acc8a8d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784902.853408:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+111:2357:x1874627658361856:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 3092us (3317us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.853421:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 481 00000100:00100000:2.0:1787784902.853967:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+111:33557:x1874627658361728:12345-192.168.203.43@tcp:57:lhsmtool_posix.0 Request processed in 9868us (10818us total) trans 4294968392 rc 0/0 00000100:00100000:2.0:1787784902.853978:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1673 00000040:00080000:1.0:1787784902.854288:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784902.854294:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784902.854924:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 11 00000040:00080000:1.0:1787784902.854935:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784902.865275:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.865425:0:5814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658362112 00000100:00100000:2.0:1787784902.865439:0:5814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658362112 00000100:00100000:2.0:1787784902.865517:0:5814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:2.0:1787784902.865530:0:5814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+110:33557:x1874627658362112:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1787784902.865900:0:5814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+111:33557:x1874627658362112:12345-192.168.203.43@tcp:35:lhsmtool_posix.0 Request processed in 370us (637us total) trans 4294968393 rc 0/0 00000100:00100000:2.0:1787784902.865911:0:5814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 347 00000100:00100000:3.0:1787784902.889689:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.890165:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658362240 00000100:00100000:2.0:1787784902.890188:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658362240 00000100:00100000:2.0:1787784902.890272:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 511 00000100:00100000:2.0:1787784902.890283:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+16:2354:x1874627658362240:12345-192.168.203.43@tcp:101:rm.0 00010000:00010000:2.0:1787784902.890307:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784902.890328:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fdb expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.890360:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=3s v=(0 0) 00010000:00010000:2.0:1787784902.890365:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784902.890376:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.890396:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 17 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.890414:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111600, rc 0) 00000100:00100000:2.0:1787784902.890488:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658362240:12345-192.168.203.43@tcp:101:rm.0 Request processed in 206us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.890496:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 511 00000100:00080000:2.0:1787784902.890538:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784902.894090:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784902.895203:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658362368 00000100:00100000:2.0:1787784902.895219:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658362368 00000100:00100000:2.0:1787784902.895305:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 512 00000100:00100000:2.0:1787784902.895316:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+36:2357:x1874627658362368:12345-192.168.203.43@tcp:101:ll_agl.0 00010000:00010000:2.0:1787784902.895340:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787784902.895365:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fe2 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.895399:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=3s v=(0 0) 00010000:00010000:2.0:1787784902.895404:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784902.895415:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 37 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.895435:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 37 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784902.895452:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989537111200, rc 0) 00000100:00080000:3.0:1787784902.895490:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784902.895529:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+37:2357:x1874627658362368:12345-192.168.203.43@tcp:101:ll_agl.0 Request processed in 215us (1454us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784902.895537:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 512 00000100:00100000:1.0:1787784902.895548:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784902.901470:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658362624 00000100:00100000:0.0:1787784902.901498:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658362624 00000100:00100000:0.0:1787784902.901666:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1674 00000100:00100000:0.0:1787784902.901678:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+120:33558:x1874627658362624:12345-192.168.203.43@tcp:36:rm.0 00000100:00100000:1.0:1787784902.901746:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:0.0:1787784902.901751:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00000100:00100000:1.0:1787784902.901889:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658362752 00000100:00100000:1.0:1787784902.901893:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658362752 00000100:00100000:1.0:1787784902.901929:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 513 00000100:00100000:1.0:1787784902.901937:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658362752:12345-192.168.203.43@tcp:101:ll_sa.0 00010000:00010000:1.0:1787784902.901949:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784902.901970:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1fe9 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.901999:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=3s v=(0 0) 00010000:00010000:1.0:1787784902.902004:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784902.902014:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 18 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.902040:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 18 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.902056:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de800, rc 0) 00000100:00100000:1.0:1787784902.902123:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2356:x1874627658362752:12345-192.168.203.43@tcp:101:ll_sa.0 Request processed in 187us (377us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784902.902130:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 513 00010000:00010000:0.0:1787784902.903309:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc7f49 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a01 expref: 120 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.903343:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:0.0:1787784902.903358:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989404ed6800/0xb0ce63137acc8a47 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fb8 expref: 119 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.903368:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784902.903371:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784902.903403:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.903418:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.903429:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.903448:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.903456:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.903463:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787784902.903874:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784902.904115:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.904129:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 110 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.904137:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.904303:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 110 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.904319:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 110 pid: 10850 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784902.904336:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668463232:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784902.904344:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784902.904407:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658362880 00000100:00100000:3.0:1787784902.904416:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658362880 00000100:00100000:3.0:1787784902.904480:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 514 00000100:00100000:3.0:1787784902.904493:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+37:2356:x1874627658362880:12345-192.168.203.43@tcp:101:ll_sa.0 00010000:00010000:3.0:1787784902.904511:0:10688:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784902.904526:0:10688:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ff0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.904553:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1787784902.904558:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784902.904568:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 38 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.904584:0:10688:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 38 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.904599:0:10688:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7000, rc 0) 00000100:00100000:0.0:1787784902.904944:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787784902.906311:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658362880:12345-192.168.203.43@tcp:101:ll_sa.0 Request processed in 1808us (1953us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.906325:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 514 00000100:00100000:1.0:1787784902.906597:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784902.906722:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784902.906745:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784902.906854:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363008 00000100:00100000:3.0:1787784902.906864:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363008 00000100:00100000:3.0:1787784902.906904:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 515 00000100:00100000:3.0:1787784902.906914:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2356:x1874627658363008:12345-192.168.203.43@tcp:101:ll_sa.0 00010000:00010000:3.0:1787784902.906935:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784902.906956:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ff7 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.906986:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1787784902.906990:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784902.906999:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.907015:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.907037:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7800, rc 0) 00000100:00100000:3.0:1787784902.908744:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+19:2356:x1874627658363008:12345-192.168.203.43@tcp:101:ll_sa.0 Request processed in 1817us (2134us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.908891:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 515 00000100:00100000:3.0:1787784902.908942:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363136 00000100:00100000:3.0:1787784902.908951:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363136 00000100:00100000:3.0:1787784902.908990:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363264 00000100:00100000:3.0:1787784902.908993:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363264 00000100:00100000:3.0:1787784902.909015:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:3.0:1787784902.909024:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658363136:12345-192.168.203.43@tcp:101:ll_sa.0 00010000:00010000:3.0:1787784902.909043:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784902.909065:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b1ffe expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.909092:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784902.909096:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784902.909104:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 39 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.909122:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 39 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.909135:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6600, rc 0) 00000100:00100000:3.0:1787784902.909185:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+39:2356:x1874627658363136:12345-192.168.203.43@tcp:101:ll_sa.0 Request processed in 162us (2462us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.909195:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:3.0:1787784902.909209:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:3.0:1787784902.909216:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+19:2356:x1874627658363264:12345-192.168.203.43@tcp:101:ll_sa.0 00010000:00010000:3.0:1787784902.909224:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784902.909234:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b2005 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.909248:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784902.909251:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784902.909258:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 20 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.909269:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 20 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.909279:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7e00, rc 0) 00000100:00100000:3.0:1787784902.909300:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+20:2356:x1874627658363264:12345-192.168.203.43@tcp:101:ll_sa.0 Request processed in 85us (2553us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.909306:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:1.0:1787784902.913545:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784902.914190:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784902.914543:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668463232:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784902.914597:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784902.915016:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363392 00000100:00100000:3.0:1787784902.915025:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363392 00000100:00100000:3.0:1787784902.915082:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363520 00000100:00100000:3.0:1787784902.915085:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363520 00000100:00100000:3.0:1787784902.915111:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:3.0:1787784902.915122:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+39:2356:x1874627658363392:12345-192.168.203.43@tcp:101:ll_sa.0 00010000:00010000:3.0:1787784902.915162:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784902.915184:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8ada lrc: 2/0,0 mode: --/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b200c expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.915214:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1787784902.915219:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784902.915230:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8ada lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b200c expref: 40 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.915252:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000113dd65f/0xb0ce63137acc8ada lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b200c expref: 40 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.915268:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed7600, rc 0) 00000100:00100000:3.0:1787784902.915333:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+40:2356:x1874627658363392:12345-192.168.203.43@tcp:101:ll_sa.0 Request processed in 212us (1803us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.915341:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:3.0:1787784902.915435:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00100000:3.0:1787784902.915498:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+20:2356:x1874627658363520:12345-192.168.203.43@tcp:101:ll_sa.0 00010000:00010000:3.0:1787784902.915508:0:6670:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787784902.915518:0:6670:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc8ae1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b2013 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.915533:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1787784902.915536:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784902.915541:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc8ae1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2013 expref: 21 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.915560:0:6670:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000091f90296/0xb0ce63137acc8ae1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2013 expref: 21 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.915568:0:6670:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff989404ed6000, rc 0) 00000100:00100000:3.0:1787784902.915588:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+21:2356:x1874627658363520:12345-192.168.203.43@tcp:101:ll_sa.0 Request processed in 93us (1918us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.915596:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00100000:3.0:1787784902.917104:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784902.922568:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363648 00000100:00100000:3.0:1787784902.922590:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363648 00010000:00010000:3.0:1787784902.922652:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 111 pid: 10850 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784902.924196:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 482 00000100:00100000:3.0:1787784902.924225:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+111:2357:x1874627658363648:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784902.924253:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784902.924267:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 111 pid: 10850 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.924288:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 111 pid: 10850 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.924321:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc7f18 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b19ec expref: 111 pid: 10850 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.924544:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:3.0:1787784902.924552:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.924637:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.924648:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.925010:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784902.925097:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+110:2357:x1874627658363648:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 885us (8002us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784902.925108:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 482 00010000:00010000:1.0:1787784902.928044:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784902.928065:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784902.930504:0:20438:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].15, 64 off9176 00000004:00080000:0.0:1787784902.931630:0:20438:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 694018479175ns, next at 695018472422ns, rc = 0 00010000:00010000:0.0:1787784902.932310:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932337:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932354:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932364:0:20438:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932372:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932378:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932405:0:20438:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787784902.932436:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932443:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932450:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932456:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784902.932461:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932469:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932488:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:0.0:1787784902.932491:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787784902.932496:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.932503:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ee00/0xb0ce63137acc8ab7 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.932717:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784902.932726:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.932732:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fc00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784902.932736:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.932742:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.932753:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff98951b73b400) 00010000:00010000:3.0:1787784902.932771:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784902.932775:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.932786:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fc00/0xb0ce63137acc8ae8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.933634:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.933652:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.933659:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.933665:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784902.933670:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.933679:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.933692:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784902.933695:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784902.933699:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784902.933704:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e200/0xb0ce63137acc8aa9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.935472:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784902.935481:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784902.935484:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:0.0:1787784902.936302:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bfaf90d5 reply state 00000000548c4eeb transno 4294968394 ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784902.936323:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784902.936488:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+120:33558:x1874627658362624:12345-192.168.203.43@tcp:36:rm.0 Request processed in 34803us (40930us total) trans 4294968394 rc 0/0 00000100:00100000:0.0:1787784902.936500:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1674 00010000:00010000:3.0:1787784902.945215:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.945242:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.945253:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.945260:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784902.945267:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.945278:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.945301:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff989520108900) 00010000:00010000:3.0:1787784902.945305:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784902.945311:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784902.945319:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e000/0xb0ce63137acc8ab0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784903.259774:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968394 is committed 00000004:00080000:3.0:1787784903.259791:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 17 committed 00000001:00080000:3.0:1787784903.259809:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968387 is committed 00000100:00100000:1.0:1787784903.260035:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668463360:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787784903.260120:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:3.0:1787784903.260347:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968390 is committed 00000001:00080000:3.0:1787784903.260456:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968393 is committed 00000100:00100000:0.0:1787784903.260496:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668463360 00000100:00100000:0.0:1787784903.260505:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668463360 00000100:00100000:3.0:1787784903.260577:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 520 00000100:00100000:3.0:1787784903.260648:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668463360:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787784903.260680:0:6670:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:29 count 1 00010000:00010000:3.0:1787784903.260709:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.260734:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 22 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.260744:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 20 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.260754:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787784903.260768:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784903.260774:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 5/0,1 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.260818:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 20 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.260834:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 20 pid: 10688 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784903.260853:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539252580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668463488:192.168.203.43@tcp:104: 00010000:00010000:3.0:1787784903.260937:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 22 pid: 6671 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.260947:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 22 pid: 6671 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784903.260956:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668463616:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784903.260975:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989538a723c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787784903.276402:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784903.276439:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784903.276477:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363904 00000100:00100000:1.0:1787784903.276490:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363904 00010000:00010000:1.0:1787784903.276560:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 23 pid: 6671 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784903.276601:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658363776 00000100:00100000:1.0:1787784903.276604:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658363776 00010000:00010000:1.0:1787784903.276623:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 21 pid: 10688 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784903.276650:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 483 00000100:00100000:1.0:1787784903.276662:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+23:2355:x1874627658363904:12345-192.168.203.43@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787784903.276671:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 484 00010000:00010000:1.0:1787784903.276685:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787784903.276686:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+21:2355:x1874627658363776:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784903.276697:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 23 pid: 6671 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.276699:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784903.276709:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 21 pid: 10688 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784903.276720:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 23 pid: 6671 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.276739:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 21 pid: 10688 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784903.277940:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539252580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668463488:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784903.277960:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6670:1874627668463616:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784903.279392:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007447218f/0xb0ce63137acc838d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c77 expref: 23 pid: 6671 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.279463:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008f95267e/0xb0ce63137acc8a94 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fdb expref: 21 pid: 10688 timeout: 794 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.279483:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784903.279490:0:5797:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787784903.279498:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.279510:0:5797:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.279544:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784903.279648:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+20:2355:x1874627658363776:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2965us (3206us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784903.279660:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 484 00010000:00010000:0.0:1787784903.289864:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784903.289914:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784903.289923:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784903.289939:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784903.289946:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784903.289995:0:6670:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787784903.290067:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784903.290078:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.290091:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784903.290097:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.290109:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.290136:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.290166:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b72ac33/0xb0ce63137acc8aef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784903.290217:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784903.290220:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784903.290222:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784903.291194:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:1.0:1787784903.291233:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784903.291339:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+22:2355:x1874627658363904:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 14671us (14947us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784903.291355:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 483 00000100:00100000:1.0:1787784903.295810:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668463360:12345-0@lo:6:osp-syn-0-0.0 Request processed in 35154us (35677us total) trans 4294967381 rc 0/0 00000100:00100000:1.0:1787784903.295829:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 520 00000004:00080000:2.0:1787784903.295971:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9894021acf00/2, rc 0, transno 4294967381 00000100:00100000:2.0:1787784903.295980:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668463360:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784903.431686:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784903.431860:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658364032 00000100:00100000:0.0:1787784903.431873:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658364032 00000100:00100000:0.0:1787784903.431943:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 619 00000100:00100000:0.0:1787784903.431955:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658364032:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784903.432069:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658364032:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 116us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784903.432077:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 619 00010000:00010000:0.0:1787784903.601522:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784903.723799:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967392 is committed 00000001:00080000:3.0:1787784903.724454:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967394 is committed 00000001:00080000:3.0:1787784903.724475:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967395 is committed 00000040:00080000:2.0:1787784903.892896:0:22799:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784903.892909:0:22799:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784903.892999:0:22799:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:2.0:1787784903.893008:0:22799:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784903.992574:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784903.992695:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658364288 00000100:00100000:2.0:1787784903.992708:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658364288 00000100:00100000:2.0:1787784903.992796:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1675 00000100:00100000:2.0:1787784903.992810:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33571:x1874627658364288:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:2.0:1787784903.992912:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784903.992949:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc801b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a55 expref: 117 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.992973:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:2.0:1787784903.992979:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784903.993011:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993027:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993038:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993055:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993063:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993070:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993796:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993812:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993822:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993875:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 59 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784903.993895:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 59 pid: 5813 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784903.993912:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668463744:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784903.994033:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff989507fd1540 going to sleep for 11 seconds 00000100:00100000:2.0:1787784904.001116:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784904.001310:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658364416 00000100:00100000:1.0:1787784904.001323:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658364416 00010000:00010000:1.0:1787784904.001395:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 60 pid: 5813 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784904.001456:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00100000:1.0:1787784904.001469:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+60:2357:x1874627658364416:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784904.001487:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784904.001495:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 60 pid: 5813 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.001512:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 60 pid: 5813 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.001540:0:21579:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:1.0:1787784904.001547:0:21579:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.001558:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.001567:0:21579:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.001591:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784904.001656:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+60:2357:x1874627658364416:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 190us (554us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784904.001665:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00080000:2.0:1787784904.004031:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784904.004581:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc7fea lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a40 expref: 59 pid: 5813 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784904.004614:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668463744:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784904.004647:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.004658:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.004664:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784904.007137:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].20, 64 off9592 00000004:00080000:0.0:1787784904.007540:0:10850:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 695094392318ns, next at 696094386804ns, rc = 0 00010000:00010000:0.0:1787784904.007666:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007695:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007711:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007722:0:10850:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007729:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007736:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007816:0:10850:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787784904.007848:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007855:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007862:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007868:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784904.007874:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007883:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007904:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:0.0:1787784904.007907:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787784904.007912:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007919:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b04 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007945:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007952:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007958:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007963:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784904.007967:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007973:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007981:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784904.007983:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784904.007987:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.007993:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110e00/0xb0ce63137acc8af6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784904.008009:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000706d9349 reply state 00000000d7d7d0ed transno 4294968395 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784904.008019:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784904.008179:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33571:x1874627658364288:12345-192.168.203.43@tcp:36:rm.0 Request processed in 15366us (15612us total) trans 4294968395 rc 0/0 00000100:00100000:0.0:1787784904.008190:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1675 00010000:00010000:1.0:1787784904.011676:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784904.011697:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.011712:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ec00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784904.011716:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.011727:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.011751:0:5798:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff989520131100) 00010000:00010000:1.0:1787784904.011766:0:5798:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787784904.011771:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.011778:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b0b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784904.021608:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784904.021620:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784904.021623:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784904.025803:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.025827:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.025843:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.025853:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784904.025859:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.025869:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.025891:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff989530cb1200) 00010000:00010000:1.0:1787784904.025894:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784904.025900:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784904.025906:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8afd lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787784904.334702:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967381 is committed 00000001:00080000:3.0:1787784904.334725:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967377 is committed 00000001:00080000:3.0:1787784904.334740:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967380 is committed 00010000:00010000:1.0:1787784904.625346:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784904.993137:0:22850:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784904.993182:0:22850:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784904.993273:0:22850:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:1.0:1787784904.993290:0:22850:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784905.075564:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784905.077624:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658364672 00000100:00100000:3.0:1787784905.077644:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658364672 00000100:00100000:3.0:1787784905.077760:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1676 00000100:00100000:3.0:1787784905.077774:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33581:x1874627658364672:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:3.0:1787784905.077871:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784905.077908:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de600/0xb0ce63137acc80ed lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1aa9 expref: 117 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.077933:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:3.0:1787784905.077939:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784905.077970:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.077987:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.077998:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078017:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078026:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078033:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078199:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078211:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 58 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078221:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078282:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 58 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.078299:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 58 pid: 5813 timeout: 796 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784905.078317:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668463872:192.168.203.43@tcp:104: 00000100:00080000:2.0:1787784905.078578:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784905.079572:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504c5bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787784905.082009:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668463872:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784905.082049:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784905.082833:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784905.082940:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658364800 00000100:00100000:3.0:1787784905.082950:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658364800 00010000:00010000:3.0:1787784905.082997:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 59 pid: 5813 timeout: 796 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784905.083044:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 486 00000100:00100000:3.0:1787784905.083055:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+59:2356:x1874627658364800:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784905.083072:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784905.083081:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 59 pid: 5813 timeout: 796 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.083095:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 59 pid: 5813 timeout: 796 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.083125:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dec00/0xb0ce63137acc80bc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1a94 expref: 59 pid: 5813 timeout: 796 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.083142:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:3.0:1787784905.083177:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.083189:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.083198:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.083226:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784905.083303:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+58:2356:x1874627658364800:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 249us (475us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784905.083313:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 486 00010000:00010000:3.0:1787784905.083407:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.083417:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787784905.084257:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].16, 64 off9240 00000004:00080000:3.0:1787784905.084270:0:5812:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 696171121684ns, next at 697171118306ns, rc = 0 00010000:00010000:3.0:1787784905.084352:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084367:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084376:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084384:0:5812:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084392:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084398:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084423:0:5812:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787784905.084448:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084455:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084462:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084469:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbd000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784905.084475:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084483:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084495:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:3.0:1787784905.084498:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787784905.084503:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084510:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbd000/0xb0ce63137acc8b20 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084532:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084538:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084544:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084549:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:3.0:1787784905.084552:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084558:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084570:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:3.0:1787784905.084573:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787784905.084576:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784905.084581:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8b12 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787784905.084592:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000034e9f9e0 reply state 000000001f21b2f4 transno 4294968396 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784905.084602:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787784905.084686:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33581:x1874627658364672:12345-192.168.203.43@tcp:36:rm.0 Request processed in 6912us (9132us total) trans 4294968396 rc 0/0 00000100:00100000:3.0:1787784905.084695:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1676 00010000:00010000:1.0:1787784905.085405:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784905.085416:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784905.085429:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784905.085433:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784905.085440:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784905.085454:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff98951b73b000) 00010000:00010000:1.0:1787784905.085456:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787784905.085460:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784905.085465:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8b27 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784905.085485:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784905.085487:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784905.085488:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784905.088735:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784905.088773:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784905.088790:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784905.088798:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784905.088803:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784905.088813:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784905.088832:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff9895198b5f00) 00010000:00010000:0.0:1787784905.088835:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784905.088841:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784905.088852:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b19 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784905.203294:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784905.203327:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668464000:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784905.203438:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784905.203528:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668464000 00000100:00100000:0.0:1787784905.203539:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668464000 00000100:00100000:0.0:1787784905.203608:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1787784905.203620:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668464000:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784905.203668:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784905.203721:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668464000:12345-0@lo:13:osp-pre-0-0.0 Request processed in 103us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784905.203729:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00080000:0.0:1787784905.203786:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784905.203822:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967381 gen 2 00000004:00080000:0.0:1787784905.203829:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9894021acf00, transno 4294967381 00000100:00100000:0.0:1787784905.203850:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668464000:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787784905.652195:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784906.527980:0:22901:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784906.527991:0:22901:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784906.528328:0:22901:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:3.0:1787784906.528401:0:22901:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784906.626359:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784906.626515:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658365056 00000100:00100000:3.0:1787784906.626524:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658365056 00000100:00100000:3.0:1787784906.626580:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1677 00000100:00100000:3.0:1787784906.626589:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33591:x1874627658365056:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:3.0:1787784906.626642:0:20438:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784906.626669:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8213 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bac expref: 117 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626687:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:3.0:1787784906.626693:0:20438:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784906.626715:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626729:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626756:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626770:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626778:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626785:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626882:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626892:0:20438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 57 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626901:0:20438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 4/0,1 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626929:0:20438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 57 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.626942:0:20438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 57 pid: 5813 timeout: 797 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784906.626957:0:20438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668464128:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784906.627135:0:20438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063ef300 going to sleep for 11 seconds 00000100:00100000:3.0:1787784906.632951:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784906.633039:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658365184 00000100:00100000:2.0:1787784906.633049:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658365184 00010000:00010000:2.0:1787784906.633104:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 58 pid: 5813 timeout: 797 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784906.633135:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 487 00000100:00100000:2.0:1787784906.633144:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+58:2357:x1874627658365184:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784906.633188:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784906.633194:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 58 pid: 5813 timeout: 797 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784906.633207:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 58 pid: 5813 timeout: 797 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784906.633234:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:2.0:1787784906.633239:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784906.633249:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784906.633257:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784906.633284:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784906.633339:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+58:2357:x1874627658365184:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 190us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784906.633347:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 487 00010000:00010000:0.0:1787784906.633958:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc81b8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1b27 expref: 57 pid: 5813 timeout: 797 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784906.633975:0:20438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539251680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:20438:1874627668464128:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784906.633995:0:20438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.634008:0:20438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.634014:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784906.635080:0:20438:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].21, 64 off9656 00000004:00080000:0.0:1787784906.635090:0:20438:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 697721942235ns, next at 698721939890ns, rc = 0 00000100:00080000:3.0:1787784906.635325:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784906.636553:0:20438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636573:0:20438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636584:0:20438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636593:0:20438:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636600:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636606:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636625:0:20438:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787784906.636645:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636650:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636657:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636664:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784906.636667:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636684:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636696:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:0.0:1787784906.636699:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787784906.636702:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636708:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3e00/0xb0ce63137acc8b3c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.636728:0:20438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.636866:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784906.636872:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.636879:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ea00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784906.636883:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.636890:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.636899:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff989520131b00) 00010000:00010000:3.0:1787784906.636901:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784906.636904:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.636910:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ea00/0xb0ce63137acc8b43 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784906.636924:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784906.636926:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784906.636928:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784906.637543:0:20438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.637550:0:20438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.637556:0:20438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784906.637559:0:20438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.637570:0:20438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.637580:0:20438:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784906.637583:0:20438:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784906.637585:0:20438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.637592:0:20438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3a00/0xb0ce63137acc8b2e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784906.637605:0:20438:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000005e4cd73 reply state 0000000020b1bd74 transno 4294968397 ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784906.637613:0:20438:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784906.637694:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33591:x1874627658365056:12345-192.168.203.43@tcp:36:rm.0 Request processed in 11103us (11340us total) trans 4294968397 rc 0/0 00000100:00100000:0.0:1787784906.637704:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1677 00010000:00010000:0.0:1787784906.643186:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.643203:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.643211:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.643217:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953f5b3400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784906.643222:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.643234:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.643249:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff9895198b5a00) 00010000:00010000:0.0:1787784906.643252:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784906.643256:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.643262:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3400/0xb0ce63137acc8b35 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784906.674836:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784907.252835:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402b13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668464256:0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.252917:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784907.252970:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6aad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668464768:0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.252997:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784907.253025:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1787784907.253032:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668464384:0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253054:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784907.253072:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668464896:0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253106:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784907.253193:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668464256 00000100:00100000:3.0:1787784907.253199:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668464256 00000100:00100000:3.0:1787784907.253242:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 620 00000100:00100000:3.0:1787784907.253251:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668464256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253299:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668464256:12345-0@lo:400:kworker.0 Request processed in 49us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784907.253305:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 620 00000100:00100000:3.0:1787784907.253335:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668464768 00000100:00100000:3.0:1787784907.253339:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668464768 00000100:00100000:3.0:1787784907.253368:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668464384 00000100:00100000:3.0:1787784907.253371:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668464384 00000100:00100000:3.0:1787784907.253389:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:3.0:1787784907.253396:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668464768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253427:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539251680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668464768:12345-0@lo:400:kworker.0 Request processed in 31us (429us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784907.253433:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:3.0:1787784907.253445:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:3.0:1787784907.253451:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668464384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253487:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668464384:12345-0@lo:400:kworker.0 Request processed in 36us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784907.253494:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:3.0:1787784907.253513:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668464896 00000100:00100000:3.0:1787784907.253516:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668464896 00000100:00100000:3.0:1787784907.253543:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1787784907.253549:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668464896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253579:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668464896:12345-0@lo:400:kworker.0 Request processed in 30us (472us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784907.253585:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00080000:3.0:1787784907.253603:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784907.253607:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784907.253654:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668464384:0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253666:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668464896:0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253674:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402b13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668464256:0@lo:400:kworker.0 00000100:00100000:3.0:1787784907.253683:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6aad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668464768:0@lo:400:kworker.0 00000100:00100000:2.0:1787784907.256579:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6ab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668464640:0@lo:400:kworker.0 00000100:00100000:2.0:1787784907.256644:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784907.256686:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787784907.256694:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668464512:0@lo:400:kworker.0 00000100:00100000:2.0:1787784907.256720:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784907.256752:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668465024:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784907.256778:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784907.256828:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668465024 00000100:00100000:2.0:1787784907.256835:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668465024 00000100:00100000:2.0:1787784907.256880:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1787784907.256888:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668465024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784907.256921:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784907.256965:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668465024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784907.256972:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1787784907.257026:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668465024:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784907.257236:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668464512 00000100:00100000:0.0:1787784907.257240:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668464512 00000100:00100000:0.0:1787784907.257280:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:0.0:1787784907.257287:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668464512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784907.257327:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668464512:12345-0@lo:400:kworker.0 Request processed in 41us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784907.257333:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:0.0:1787784907.257359:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668464640 00000100:00100000:0.0:1787784907.257362:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668464640 00000100:00100000:0.0:1787784907.257385:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1787784907.257390:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668464640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784907.257418:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668464640:12345-0@lo:400:kworker.0 Request processed in 28us (774us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784907.257423:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00080000:0.0:1787784907.257443:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784907.257447:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784907.257466:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668464512:0@lo:400:kworker.0 00000100:00100000:0.0:1787784907.257475:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6ab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668464640:0@lo:400:kworker.0 00010000:00010000:2.0:1787784907.697293:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784907.783588:0:22952:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784907.783599:0:22952:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784907.783670:0:22952:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:1.0:1787784907.783685:0:22952:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784907.883906:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784907.884027:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658365440 00000100:00100000:2.0:1787784907.884039:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658365440 00000100:00100000:2.0:1787784907.884140:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1678 00000100:00100000:2.0:1787784907.884174:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:33601:x1874627658365440:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:2.0:1787784907.884256:0:5812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784907.884293:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f600/0xb0ce63137acc82fa lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c1c expref: 117 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884320:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:2.0:1787784907.884326:0:5812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784907.884357:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884383:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884395:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884413:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884420:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884435:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884581:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884592:0:5812:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 56 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884601:0:5812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884640:0:5812:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 56 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.884667:0:5812:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 56 pid: 5811 timeout: 798 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784907.884689:0:5812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668465152:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784907.884938:0:5812:0:(client.c:2715:ptlrpc_set_wait()) set ffff98954088f300 going to sleep for 11 seconds 00000100:00080000:3.0:1787784907.885091:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784907.915391:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784907.915530:0:5817:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658365568 00000100:00100000:0.0:1787784907.915544:0:5817:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658365568 00000100:00100000:0.0:1787784907.915613:0:5817:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 91 00000100:00100000:0.0:1787784907.915625:0:5817:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:2356:x1874627658365568:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:0.0:1787784907.915673:0:5817:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787784907.915700:0:5817:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8b5f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b201a expref: -99 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915804:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915823:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915841:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915859:0:5817:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915866:0:5817:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915873:0:5817:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915962:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915975:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915983:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.915989:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784907.915994:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916012:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916027:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784907.916031:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784907.916037:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916043:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e600/0xb0ce63137acc8b66 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916059:0:5817:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916064:0:5817:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916069:0:5817:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916074:0:5817:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ec00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784907.916078:0:5817:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916084:0:5817:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916091:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xbb834312 (ffff989520108000) 00010000:00010000:0.0:1787784907.916093:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784907.916096:0:5817:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916102:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704ec00/0xb0ce63137acc8b6d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5817 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916142:0:5817:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8b5f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b201a expref: 118 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916177:0:5817:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784907.916180:0:5817:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787784907.916183:0:5817:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704e400/0xb0ce63137acc8b5f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b201a expref: 118 pid: 5817 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.916193:0:5817:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff98953704e400, rc -2) 00000100:00100000:0.0:1787784907.916271:0:5817:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+117:2356:x1874627658365568:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 648us (894us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787784907.916280:0:5817:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 91 00000100:00100000:1.0:1787784907.921054:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784907.921355:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658365696 00000100:00100000:1.0:1787784907.921366:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658365696 00000100:00100000:1.0:1787784907.921444:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 521 00000100:00100000:1.0:1787784907.921456:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+40:2354:x1874627658365696:12345-192.168.203.43@tcp:101:ll_agl.0 00010000:00010000:1.0:1787784907.921479:0:6669:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784907.921503:0:6669:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000689e8511/0xb0ce63137acc8b74 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b202f expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784907.921535:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=6s v=(0 0) 00010000:00010000:1.0:1787784907.921541:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x24:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784907.921553:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000689e8511/0xb0ce63137acc8b74 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b202f expref: 41 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784907.921576:0:6669:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000689e8511/0xb0ce63137acc8b74 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x24:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b202f expref: 41 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784907.921593:0:6669:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de600, rc 0) 00000100:00100000:1.0:1787784907.921650:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+41:2354:x1874627658365696:12345-192.168.203.43@tcp:101:ll_agl.0 Request processed in 197us (613us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784907.921658:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 521 00000100:00100000:2.0:1787784907.925536:0:5812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5812:1874627668465152:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784907.925579:0:5812:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784907.926976:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:3.0:1787784907.927268:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784907.927464:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658365824 00000100:00100000:2.0:1787784907.927477:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658365824 00010000:00010000:2.0:1787784907.927528:0:5797:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 57 pid: 5811 timeout: 798 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784907.927581:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 488 00000100:00100000:2.0:1787784907.927592:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+57:2357:x1874627658365824:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784907.927611:0:5797:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784907.927619:0:5797:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 57 pid: 5811 timeout: 798 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.927632:0:5797:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 57 pid: 5811 timeout: 798 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.927659:0:5797:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f000/0xb0ce63137acc82c9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1bf2 expref: 57 pid: 5811 timeout: 798 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.927676:0:5797:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:2.0:1787784907.927682:0:5797:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.927693:0:5797:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.927702:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.927751:0:5797:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784907.927939:0:5812:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.927953:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787784907.929131:0:5812:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].17, 64 off9304 00000004:00080000:0.0:1787784907.929181:0:5812:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 699016032954ns, next at 700015994664ns, rc = 0 00010000:00010000:0.0:1787784907.929262:0:5812:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929278:0:5812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929289:0:5812:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929297:0:5812:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929304:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929310:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929328:0:5812:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787784907.929353:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929359:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929364:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929371:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784907.929376:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929385:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929397:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:0.0:1787784907.929399:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787784907.929404:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929409:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8b58 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929430:0:5812:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929435:0:5812:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929440:0:5812:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929444:0:5812:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110c00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784907.929447:0:5812:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929452:0:5812:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929461:0:5812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784907.929463:0:5812:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784907.929466:0:5812:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784907.929472:0:5812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8b4a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784907.929481:0:5812:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a5d9bb52 reply state 00000000f301ae71 transno 4294968398 ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784907.929490:0:5812:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784907.929618:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+56:2357:x1874627658365824:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 2023us (2645us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784907.929629:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 488 00010000:00010000:3.0:1787784907.930233:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784907.930241:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784907.930249:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704f800 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784907.930253:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784907.930261:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784907.930274:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff989520108600) 00010000:00010000:3.0:1787784907.930276:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784907.930280:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 1/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784907.930285:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f800/0xb0ce63137acc8b7b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784907.930308:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784907.930309:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784907.930311:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784907.934304:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539252580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+118:33601:x1874627658365440:12345-192.168.203.43@tcp:36:rm.0 Request processed in 50130us (50387us total) trans 4294968398 rc 0/0 00000100:00100000:1.0:1787784907.934321:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1678 00010000:00010000:2.0:1787784907.942602:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.942624:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.942633:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.942639:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784907.942646:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.942656:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.942678:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff989530cb1500) 00010000:00010000:2.0:1787784907.942682:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784907.942688:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784907.942695:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110000/0xb0ce63137acc8b51 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5812 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787784908.313185:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968398 is committed 00000004:00080000:2.0:1787784908.313204:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 19 committed 00000004:00080000:2.0:1787784908.313229:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 23 committed 00000004:00080000:2.0:1787784908.313247:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 22 committed 00000004:00080000:2.0:1787784908.313254:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 18 committed 00000100:00100000:0.0:1787784908.313459:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668465408:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1787784908.313551:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784908.313655:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668465408 00000100:00100000:0.0:1787784908.313663:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668465408 00000100:00100000:0.0:1787784908.313719:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:0.0:1787784908.313742:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668465408:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784908.313860:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784908.313872:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668465280:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784908.313907:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784908.314309:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784908.314321:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402184b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668465664:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1787784908.314356:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784908.314386:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668465280 00000100:00100000:1.0:1787784908.314390:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668465280 00000100:00100000:1.0:1787784908.314414:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668465664 00000100:00100000:1.0:1787784908.314417:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668465664 00000100:00100000:1.0:1787784908.314435:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 523 00000100:00100000:1.0:1787784908.314442:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668465280:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1787784908.314475:0:10688:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:30 count 1 00010000:00010000:1.0:1787784908.314517:0:10688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.314545:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.314556:0:10688:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 19 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.314566:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=6s v=(0 0) 00010000:00010000:1.0:1787784908.314571:0:10688:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784908.314577:0:10688:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784908.314582:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784908.314591:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402185e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668465536:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1787784908.314608:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 19 pid: 6669 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784908.314617:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787784908.314626:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 19 pid: 6669 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784908.314639:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508851a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668465792:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784908.314742:0:10688:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 11 pid: 13825 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.314760:0:10688:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 11 pid: 13825 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784908.314770:0:10688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668465920:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784908.314794:0:10688:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953079aa80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784908.314811:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668465536 00000100:00100000:1.0:1787784908.314815:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668465536 00000100:00100000:1.0:1787784908.314836:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 524 00000100:00100000:1.0:1787784908.314843:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668465664:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1787784908.314856:0:6669:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:33 count 1 00002000:00080000:0.0:1787784908.315003:0:6671:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:31 count 1 00010000:00010000:0.0:1787784908.315019:0:6671:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315033:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315042:0:6671:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315054:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=6s v=(0 0) 00010000:00010000:0.0:1787784908.315058:0:6671:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784908.315061:0:6671:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315083:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315093:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784908.315106:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668466048:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784908.315186:0:6671:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 11 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315197:0:6671:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 11 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784908.315206:0:6671:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401837840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668466176:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784908.315229:0:6671:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a56c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787784908.315251:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1787784908.315261:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668465536:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1787784908.315273:0:13825:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:32 count 1 00010000:00010000:0.0:1787784908.315293:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315307:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 14 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315317:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 40 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315324:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=6s v=(0 0) 00010000:00010000:0.0:1787784908.315327:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784908.315330:0:6669:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315331:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.315343:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 14 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.315351:0:6669:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 40 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315353:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 40 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.315359:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=6s v=(0 0) 00010000:00010000:1.0:1787784908.315362:0:6669:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784908.315363:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 40 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.315366:0:6669:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784908.315374:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668466304:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784908.315390:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 40 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.315399:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 40 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.315405:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 14 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784908.315409:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668466432:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784908.315416:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 14 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784908.315432:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401837c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668466560:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784908.315437:0:6669:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 14 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.315445:0:6669:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 14 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784908.315449:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895401a5780 going to sleep for 11 seconds 00000100:00100000:1.0:1787784908.315455:0:6669:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035da940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668466688:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784908.315475:0:6669:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953079ad80 going to sleep for 11 seconds 00000100:00100000:1.0:1787784908.319001:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508851a40 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668465792:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784908.319024:0:10688:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10688:1874627668465920:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784908.319064:0:10688:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784908.326117:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784908.326209:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784908.326369:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658365952 00000100:00100000:2.0:1787784908.326394:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658365952 00010000:00010000:2.0:1787784908.326462:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 20 pid: 6669 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784908.326511:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366080 00000100:00100000:2.0:1787784908.326515:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366080 00010000:00010000:2.0:1787784908.326532:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 12 pid: 13825 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784908.326556:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 489 00000100:00100000:2.0:1787784908.326567:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+20:2356:x1874627658365952:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784908.326586:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784908.326592:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 20 pid: 6669 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326608:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 20 pid: 6669 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326637:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081a0022d/0xb0ce63137acc8aa2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe9 expref: 20 pid: 6669 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326659:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784908.326736:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+19:2356:x1874627658365952:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 160us (626us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784908.326745:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 489 00000100:00100000:2.0:1787784908.326764:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 490 00000100:00100000:2.0:1787784908.326771:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658366080:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784908.326782:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784908.326787:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 12 pid: 13825 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326797:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 12 pid: 13825 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326813:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fefb50b3/0xb0ce63137acc8386 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c70 expref: 12 pid: 13825 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326825:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784908.326829:0:5795:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784908.326836:0:5795:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326849:0:5795:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.326902:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784908.326923:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2356:x1874627658366080:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 152us (713us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784908.326930:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 490 00010000:00010000:0.0:1787784908.326937:0:10688:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.326948:0:10688:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.326959:0:10688:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.326967:0:10688:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.326987:0:10688:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00080000:3.0:1787784908.327218:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787784908.327225:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784908.327234:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.327244:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393de200 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784908.327249:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.327259:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.327271:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.327279:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5bc472c/0xb0ce63137acc8b82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10688 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.327301:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784908.327303:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784908.327304:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784908.328780:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3276:x1874627668465280:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14331us (14865us total) trans 4294967382 rc 0/0 00000100:00100000:0.0:1787784908.328795:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 523 00000004:00080000:1.0:1787784908.328954:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401836940/2, rc 0, transno 4294967382 00000100:00100000:1.0:1787784908.328963:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668465280:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787784908.347993:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784908.348032:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784908.348041:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668466048:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784908.348060:0:6671:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401837840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6671:1874627668466176:192.168.203.43@tcp:104: 00000100:00100000:3.0:1787784908.348068:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00010000:00010000:0.0:1787784908.348102:0:6671:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784908.348137:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366336 00000100:00100000:3.0:1787784908.348183:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366336 00000100:00100000:0.0:1787784908.348218:0:5795:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366208 00000100:00100000:0.0:1787784908.348223:0:5795:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366208 00010000:00010000:3.0:1787784908.348233:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 11 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348247:0:5795:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784908.348284:0:21579:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366464 00000100:00100000:3.0:1787784908.348289:0:21579:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366464 00010000:00010000:3.0:1787784908.348319:0:21579:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 41 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784908.348334:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668466304:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784908.348358:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668466432:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784908.348583:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 491 00000100:00100000:0.0:1787784908.348598:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+11:2357:x1874627658366336:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784908.348620:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784908.348628:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 11 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348642:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 11 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348672:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a2a2eaa/0xb0ce63137acc84eb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cee expref: 11 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348695:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784908.348737:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 492 00000100:00100000:3.0:1787784908.348748:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+19:2357:x1874627658366208:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784908.348759:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784908.348765:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784908.348779:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784908.348785:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2357:x1874627658366336:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 187us (806us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784908.348799:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 491 00010000:00010000:3.0:1787784908.348805:0:21579:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ac822b5/0xb0ce63137acc8ac5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff7 expref: 19 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784908.348822:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00000100:00100000:0.0:1787784908.348825:0:5795:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 493 00010000:00010000:3.0:1787784908.348829:0:21579:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00000100:00100000:0.0:1787784908.348836:0:5795:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+41:2357:x1874627658366464:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784908.348838:0:21579:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348848:0:5795:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784908.348853:0:5795:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 41 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348865:0:5795:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 41 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348880:0:5795:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003af73581/0xb0ce63137acc8a9b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1fe2 expref: 41 pid: 6670 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.348892:0:5795:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784908.348910:0:5795:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+40:2357:x1874627658366464:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 77us (846us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784908.348916:0:5795:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 493 00010000:00010000:3.0:1787784908.348990:0:21579:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784908.349025:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787784908.354219:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2357:x1874627658366208:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 5455us (6167us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784908.354238:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 492 00010000:00010000:0.0:1787784908.354336:0:6671:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.354356:0:6671:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.354371:0:6671:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.354381:0:6671:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784908.354422:0:6671:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787784908.354470:0:5798:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784908.354481:0:5798:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.354496:0:5798:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704ec00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784908.354513:0:5798:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.354526:0:5798:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.354552:0:5798:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.354561:0:5798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027190a8a/0xb0ce63137acc8b89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6671 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.354591:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784908.354593:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784908.354595:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784908.355686:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3278:x1874627668465408:12345-0@lo:6:osp-syn-0-0.0 Request processed in 41939us (42128us total) trans 4294967383 rc 0/0 00000100:00100000:0.0:1787784908.355700:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000004:00080000:3.0:1787784908.357047:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401836d00/2, rc 0, transno 4294967383 00000100:00100000:3.0:1787784908.357062:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668465408:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787784908.372694:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784908.372772:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784908.372800:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784908.372836:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366592 00000100:00100000:2.0:1787784908.372847:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366592 00010000:00010000:2.0:1787784908.372895:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 40 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784908.372933:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366720 00000100:00100000:2.0:1787784908.372937:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366720 00010000:00010000:2.0:1787784908.372965:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 15 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784908.372982:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366848 00000100:00100000:2.0:1787784908.372985:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366848 00010000:00010000:2.0:1787784908.372999:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 16 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784908.373014:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 494 00000100:00100000:2.0:1787784908.373025:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+40:2356:x1874627658366592:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784908.373043:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784908.373049:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 40 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373067:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 40 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373097:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e6f9c0d7/0xb0ce63137acc8abe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ff0 expref: 40 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373123:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784908.373237:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+39:2356:x1874627658366592:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 212us (556us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784908.373245:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 494 00000100:00100000:2.0:1787784908.373267:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 495 00000100:00100000:2.0:1787784908.373274:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2356:x1874627658366720:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784908.373283:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784908.373287:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 16 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373297:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 16 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373313:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784908.373319:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784908.373327:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373339:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373365:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784908.373383:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+16:2356:x1874627658366720:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 110us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784908.373389:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 495 00000100:00100000:2.0:1787784908.373453:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 496 00000100:00100000:2.0:1787784908.373460:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658366848:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787784908.373474:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784908.373478:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 15 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373497:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 15 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373510:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784908.373513:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784908.373518:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373527:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784908.373542:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787784908.373565:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+15:2357:x1874627658366848:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 105us (763us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784908.373572:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 496 00010000:00010000:1.0:1787784908.373721:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aea66ba3/0xb0ce63137acc8394 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1c7e expref: 14 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784908.373751:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401837c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13825:1874627668466560:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784908.373862:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787784908.373954:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787784908.373963:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.373971:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.373983:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.373991:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.374015:0:13825:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787784908.375255:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3278:x1874627668465536:12345-0@lo:6:osp-syn-1-0.0 Request processed in 59990us (60630us total) trans 4294967396 rc 0/0 00000100:00100000:1.0:1787784908.375275:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00010000:00010000:1.0:1787784908.375317:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784908.375325:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375337:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704e600 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784908.375341:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375355:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375373:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375382:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c4d5e170/0xb0ce63137acc8b90 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375404:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784908.375406:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784908.375407:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784908.375446:0:6669:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002f07b9e/0xb0ce63137acc84f2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cf5 expref: 13 pid: 10688 timeout: 799 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784908.375457:0:6669:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035da940 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6669:1874627668466688:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784908.375481:0:6669:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375491:0:6669:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375499:0:6669:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375514:0:6669:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375522:0:6669:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.375609:0:6669:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787784908.376133:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3277:x1874627668465664:12345-0@lo:6:osp-syn-1-0.0 Request processed in 61288us (61774us total) trans 4294967397 rc 0/0 00000100:00100000:1.0:1787784908.376141:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 524 00010000:00010000:1.0:1787784908.376198:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787784908.376204:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.376214:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dfc00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:1.0:1787784908.376218:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.376226:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.376240:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.376249:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e223e7e/0xb0ce63137acc8b97 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6669 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784908.376266:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784908.376268:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784908.376270:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787784908.376305:0:3277:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402184b40/2, rc 0, transno 4294967397 00000100:00100000:1.0:1787784908.376312:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402184b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668465664:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1787784908.376362:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402185e00/2, rc 0, transno 4294967396 00000100:00100000:1.0:1787784908.376367:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402185e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668465536:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1787784908.563093:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784908.563224:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658366976 00000100:00100000:2.0:1787784908.563238:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658366976 00000100:00100000:2.0:1787784908.563305:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 621 00000100:00100000:2.0:1787784908.563318:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658366976:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784908.563428:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658366976:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 113us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784908.563437:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 621 00010000:00010000:2.0:1787784908.725518:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1787784908.943086:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967397 is committed 00000040:00080000:0.0:1787784909.512419:0:23005:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784909.512433:0:23005:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784909.512516:0:23005:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:0.0:1787784909.512523:0:23005:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:3.0:1787784909.540289:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967383 is committed 00000100:00100000:3.0:1787784909.666941:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784909.669359:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784909.674765:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658367360 00000100:00100000:3.0:1787784909.674789:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658367360 00000100:00100000:3.0:1787784909.674915:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1679 00000100:00100000:3.0:1787784909.674929:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33614:x1874627658367360:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:3.0:1787784909.675010:0:10850:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787784909.675049:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de400/0xb0ce63137acc850e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1cfc expref: 113 pid: 20438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675076:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:3.0:1787784909.675083:0:10850:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787784909.675115:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675143:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675182:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675200:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675207:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675214:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787784909.675467:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787784909.675809:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,1 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675823:0:10850:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675833:0:10850:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 4/0,1 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675875:0:10850:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 55 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.675893:0:10850:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 55 pid: 5811 timeout: 800 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784909.675911:0:10850:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6a92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668466816:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784909.676109:0:5816:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658367104 00000100:00100000:1.0:1787784909.676117:0:5816:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658367104 00000100:00100000:1.0:1787784909.676184:0:5816:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 92 00000100:00100000:1.0:1787784909.676194:0:5816:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:2357:x1874627658367104:12345-192.168.203.43@tcp:63:ll_sa.0 00010000:00010000:1.0:1787784909.676222:0:5816:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787784909.676235:0:5816:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc8bb3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xda2ac8b3841b2036 expref: -99 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676278:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676288:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676298:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676313:0:5816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676325:0:5816:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676332:0:5816:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676507:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676516:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676523:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676530:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393df000 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784909.676535:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676543:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676555:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784909.676558:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784909.676563:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676570:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df000/0xb0ce63137acc8bba lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676582:0:5816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676589:0:5816:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676594:0:5816:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676599:0:5816:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9895393dee00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784909.676602:0:5816:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676608:0:5816:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676614:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x4b66ad81 (ffff98951b73be00) 00010000:00010000:1.0:1787784909.676617:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784909.676620:0:5816:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676626:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393dee00/0xb0ce63137acc8bc1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5816 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676659:0:5816:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc8bb3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2036 expref: 114 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676668:0:5816:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:1.0:1787784909.676671:0:5816:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787784909.676675:0:5816:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393de200/0xb0ce63137acc8bb3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2036 expref: 114 pid: 5816 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784909.676691:0:5816:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9895393de200, rc -2) 00000100:00100000:3.0:1787784909.676916:0:10850:0:(client.c:2715:ptlrpc_set_wait()) set ffff98953051d600 going to sleep for 11 seconds 00000100:00100000:1.0:1787784909.694097:0:5816:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6ab840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:2357:x1874627658367104:12345-192.168.203.43@tcp:63:ll_sa.0 Request processed in 17886us (27152us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787784909.694117:0:5816:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 92 00000100:00100000:0.0:1787784909.743332:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784909.743375:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658367488 00000100:00100000:0.0:1787784909.743387:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658367488 00010000:00010000:0.0:1787784909.743455:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 56 pid: 5811 timeout: 800 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784909.743497:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 497 00000100:00100000:0.0:1787784909.743509:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:2357:x1874627658367488:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784909.743529:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784909.743537:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 56 pid: 5811 timeout: 800 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.743553:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 56 pid: 5811 timeout: 800 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.743581:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:0.0:1787784909.743587:0:5796:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,1 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.743604:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.743613:0:5796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.743636:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784909.743696:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+56:2357:x1874627658367488:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 188us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784909.743704:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 497 00000100:00080000:0.0:1787784909.743749:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784909.744111:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8497 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ccb expref: 55 pid: 5811 timeout: 800 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784909.744129:0:10850:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6a92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:10850:1874627668466816:192.168.203.43@tcp:104: 00010000:00010000:0.0:1787784909.744180:0:10850:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.744191:0:10850:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.744196:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.746072:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784909.750178:0:10850:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].22, 64 off9720 00000004:00080000:0.0:1787784909.750202:0:10850:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 700837054042ns, next at 701837049885ns, rc = 0 00010000:00010000:0.0:1787784909.750326:0:10850:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750353:0:10850:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750367:0:10850:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750377:0:10850:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750385:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750392:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750418:0:10850:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787784909.750446:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750452:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750459:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750464:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989401dbc400 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784909.750469:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750477:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750496:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:0.0:1787784909.750499:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787784909.750504:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 2/0,0 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.750510:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401dbc400/0xb0ce63137acc8bac lrc: 0/0,0 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.750906:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784909.750916:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.750924:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fa00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784909.750927:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.750933:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.750942:0:5799:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff989520131700) 00010000:00010000:3.0:1787784909.750945:0:5799:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787784909.750948:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.750953:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704fa00/0xb0ce63137acc8bc8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784909.750969:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784909.750971:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784909.750973:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784909.758909:0:10850:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.758921:0:10850:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.758929:0:10850:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.758936:0:10850:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150a200 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784909.758942:0:10850:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.758951:0:10850:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.758972:0:10850:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:0.0:1787784909.758975:0:10850:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784909.758981:0:10850:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.758988:0:10850:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150a200/0xb0ce63137acc8b9e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787784909.759006:0:10850:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e4c9c250 reply state 000000006a38616d transno 4294968399 ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784909.759016:0:10850:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787784909.759174:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33614:x1874627658367360:12345-192.168.203.43@tcp:36:rm.0 Request processed in 84241us (89821us total) trans 4294968399 rc 0/0 00000100:00100000:0.0:1787784909.759186:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1679 00010000:00010000:0.0:1787784909.762948:0:3262:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.762972:0:3262:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.762981:0:3262:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.762987:0:3262:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98950150b600 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:0.0:1787784909.762992:0:3262:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.763001:0:3262:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.763021:0:3262:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff9895198b5300) 00010000:00010000:0.0:1787784909.763024:0:3262:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787784909.763030:0:3262:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784909.763036:0:3262:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950150b600/0xb0ce63137acc8ba5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10850 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784910.329252:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668466944:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784910.329353:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784910.329740:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668466944 00000100:00100000:2.0:1787784910.329751:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668466944 00000100:00100000:2.0:1787784910.329842:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1787784910.329853:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668466944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787784910.329896:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784910.329958:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668466944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 107us (604us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784910.329965:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:1.0:1787784910.330224:0:3277:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967383 gen 2 00000004:00080000:1.0:1787784910.330231:0:3277:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401836940, transno 4294967382 00000004:00080000:1.0:1787784910.330249:0:3277:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401836d00, transno 4294967383 00000100:00100000:1.0:1787784910.330263:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668466944:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784910.330385:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784910.778778:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784911.479072:0:23057:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784911.479087:0:23057:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784911.479197:0:23057:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:2.0:1787784911.479207:0:23057:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784911.598231:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784911.598504:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658367744 00000100:00100000:2.0:1787784911.598519:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658367744 00000100:00100000:2.0:1787784911.598607:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 1680 00000100:00100000:2.0:1787784911.598620:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+113:33624:x1874627658367744:12345-192.168.203.43@tcp:36:rm.0 00010000:00010000:2.0:1787784911.598712:0:5813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787784911.598750:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9895393df400/0xb0ce63137acc8618 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d5e expref: 113 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.598771:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784911.598777:0:5813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784911.598807:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.598824:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.598840:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.598856:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.598863:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.598871:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.599018:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.599029:0:5813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 54 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.599038:0:5813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 4/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.599082:0:5813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 54 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.599098:0:5813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 54 pid: 5812 timeout: 802 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784911.599114:0:5813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668467072:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784911.599246:0:5813:0:(client.c:2715:ptlrpc_set_wait()) set ffff989506677d80 going to sleep for 11 seconds 00000100:00080000:2.0:1787784911.599267:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784911.607125:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784911.607284:0:5813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5813:1874627668467072:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784911.607321:0:5813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784911.619599:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658367872 00000100:00100000:1.0:1787784911.619617:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658367872 00000100:00080000:2.0:1787784911.619775:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787784911.620070:0:11303:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 55 pid: 5812 timeout: 802 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784911.620122:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 498 00000100:00100000:1.0:1787784911.620134:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+55:2357:x1874627658367872:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784911.620183:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784911.620193:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 55 pid: 5812 timeout: 802 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.620209:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 55 pid: 5812 timeout: 802 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.620243:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953704f200/0xb0ce63137acc85e7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1d49 expref: 55 pid: 5812 timeout: 802 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.620262:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:1.0:1787784911.620269:0:11303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.620279:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.620288:0:11303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.620320:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787784911.620405:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658367872:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 273us (13295us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784911.620414:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 498 00010000:00010000:2.0:1787784911.620528:0:5813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.620536:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787784911.622573:0:5813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].18, 64 off9368 00000004:00080000:2.0:1787784911.622597:0:5813:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 702709448975ns, next at 703709445043ns, rc = 0 00010000:00010000:2.0:1787784911.622688:0:5813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622723:0:5813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622737:0:5813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622747:0:5813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622754:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622760:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622796:0:5813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787784911.622822:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622828:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622835:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622841:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111800 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784911.622846:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622854:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622867:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784911.622870:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787784911.622876:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 2/0,0 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622883:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111800/0xb0ce63137acc8bdd lrc: 0/0,0 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622905:0:5813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622911:0:5813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622921:0:5813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622925:0:5813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111e00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:2.0:1787784911.622928:0:5813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622934:0:5813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622942:0:5813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff989520108300) 00010000:00010000:2.0:1787784911.622945:0:5813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787784911.622948:0:5813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.622954:0:5813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111e00/0xb0ce63137acc8bcf lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787784911.622963:0:5813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000718cb3f1 reply state 00000000dea22d97 transno 4294968400 ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784911.622973:0:5813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787784911.623082:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+114:33624:x1874627658367744:12345-192.168.203.43@tcp:36:rm.0 Request processed in 24460us (24859us total) trans 4294968400 rc 0/0 00000100:00100000:2.0:1787784911.623092:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 1680 00010000:00010000:2.0:1787784911.623143:0:17918:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787784911.623174:0:17918:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.623182:0:17918:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537110c00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:2.0:1787784911.623186:0:17918:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.623194:0:17918:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.623203:0:17918:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff989520108b00) 00010000:00010000:2.0:1787784911.623205:0:17918:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787784911.623208:0:17918:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.623214:0:17918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537110c00/0xb0ce63137acc8be4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784911.623234:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784911.623236:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784911.623237:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787784911.642937:0:3263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.642961:0:3263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.642970:0:3263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.642978:0:3263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111a00 already unused, calling callback (ffffffffc21c34f0) 00010000:00010000:1.0:1787784911.642984:0:3263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.642998:0:3263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.643031:0:3263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff989530cb1900) 00010000:00010000:1.0:1787784911.643035:0:3263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787784911.643040:0:3263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.643048:0:3263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989537111a00/0xb0ce63137acc8bd6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784911.795823:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784912.055906:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784912.055920:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784912.057275:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 2 00000040:00080000:0.0:1787784912.057293:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784912.369420:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784912.369487:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784912.369504:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668467328:0@lo:400:kworker.0 00000100:00100000:0.0:1787784912.369505:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668467200:0@lo:400:kworker.0 00000100:00100000:1.0:1787784912.369673:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508851a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668467456:0@lo:400:kworker.0 00000004:00080000:2.0:1787784912.370268:0:8078:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 703457118779ns, next at 704457115981ns, rc = 0 00000100:00100000:2.0:1787784912.370692:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784912.370770:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784912.370797:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668467328 00000100:00100000:3.0:1787784912.370808:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668467328 00000100:00100000:3.0:1787784912.370869:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:3.0:1787784912.370881:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668467328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.371008:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539129e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668467712:0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.371047:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784912.371070:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784912.371076:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668467840:0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.371105:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784912.371169:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668467712 00000100:00100000:2.0:1787784912.371174:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668467712 00000100:00100000:2.0:1787784912.371217:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1787784912.371223:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784912.371228:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668467712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784912.371265:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668467968:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784912.371282:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402b13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668467712:12345-0@lo:400:kworker.0 Request processed in 57us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784912.371299:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1787784912.371299:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784912.371323:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668467456 00000100:00100000:1.0:1787784912.371324:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784912.371328:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668467456 00000100:00100000:1.0:1787784912.371333:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668467584:0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.371357:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:1.0:1787784912.371362:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784912.371367:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668467456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784912.371402:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668467840 00000100:00100000:1.0:1787784912.371407:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668467840 00000100:00100000:2.0:1787784912.371416:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668467456:12345-0@lo:400:kworker.0 Request processed in 51us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784912.371425:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:1.0:1787784912.371442:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668467584 00000100:00100000:1.0:1787784912.371446:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668467584 00000100:00100000:2.0:1787784912.371454:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1787784912.371472:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:1.0:1787784912.371480:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668467584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784912.371512:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668467584:12345-0@lo:400:kworker.0 Request processed in 33us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784912.371521:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:2.0:1787784912.371761:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668467840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.371797:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668467840:12345-0@lo:400:kworker.0 Request processed in 335us (691us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784912.371804:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00080000:2.0:1787784912.371826:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784912.371830:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784912.371834:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784912.371852:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668467968 00000100:00100000:2.0:1787784912.371856:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668467968 00000100:00100000:2.0:1787784912.371884:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1787784912.371890:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668467968:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784912.371924:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784912.371960:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668467968:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (665us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784912.371966:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1787784912.372007:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668467584:0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.372019:0:3276:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967397 gen 2 00000004:00080000:2.0:1787784912.372025:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402185e00, transno 4294967396 00000004:00080000:2.0:1787784912.372035:0:3276:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402184b40, transno 4294967397 00000100:00100000:2.0:1787784912.372045:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668467968:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784912.372057:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508851a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668467456:0@lo:400:kworker.0 00000100:00100000:0.0:1787784912.372202:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668467200 00000100:00100000:0.0:1787784912.372207:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668467200 00000100:00100000:0.0:1787784912.372237:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:0.0:1787784912.372245:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668467200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784912.372284:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668467200:12345-0@lo:400:kworker.0 Request processed in 41us (1514us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784912.372291:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:0.0:1787784912.372349:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668467200:0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.373463:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668467840:0@lo:400:kworker.0 00000100:00100000:2.0:1787784912.373484:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539129e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668467712:0@lo:400:kworker.0 00000100:00100000:3.0:1787784912.373670:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668467328:12345-0@lo:400:kworker.0 Request processed in 2788us (2976us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784912.373685:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1787784912.373778:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668467328:0@lo:400:kworker.0 00000040:00080000:3.0:1787784912.424649:0:23108:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784912.424662:0:23108:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784912.424728:0:23108:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 2 00000040:00080000:3.0:1787784912.424735:0:23108:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784912.817342:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787784913.496457:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968399 is committed 00000004:00080000:2.0:1787784913.496473:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 24 committed 00000004:00080000:2.0:1787784913.496498:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 25 committed 00000001:00080000:2.0:1787784913.496504:0:5833:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968400 is committed 00000004:00080000:2.0:1787784913.496506:0:5833:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 20 committed 00000100:00100000:2.0:1787784913.496659:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784913.496700:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402185e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668468096:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787784913.496787:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784913.497303:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668468224:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787784913.497338:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784913.510280:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668468096 00000100:00100000:0.0:1787784913.510300:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668468096 00000100:00100000:0.0:1787784913.510385:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668468224 00000100:00100000:0.0:1787784913.510388:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668468224 00000100:00100000:0.0:1787784913.510419:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:0.0:1787784913.510431:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668468096:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1787784913.510472:0:13825:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:32 count 1 00010000:00010000:0.0:1787784913.510503:0:13825:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.510532:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.510543:0:13825:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.510553:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=11s v=(0 0) 00010000:00010000:0.0:1787784913.510557:0:13825:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784913.510563:0:13825:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 5/0,1 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.510605:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 17 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.510623:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 17 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784913.510639:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401837480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668468352:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784913.510779:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 527 00000100:00100000:2.0:1787784913.510791:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668468224:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1787784913.510805:0:6670:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:34 count 1 00010000:00010000:2.0:1787784913.510821:0:6670:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.510835:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 12 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.510852:0:6670:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 38 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.510860:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=11s v=(0 0) 00010000:00010000:2.0:1787784913.510863:0:6670:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787784913.510877:0:6670:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.510901:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 38 pid: 6670 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.510912:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 38 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784913.510924:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668468480:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784913.511397:0:6670:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 12 pid: 22706 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.511409:0:6670:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 12 pid: 22706 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784913.511420:0:6670:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989508852d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668468608:192.168.203.43@tcp:104: 00000100:00100000:2.0:1787784913.511444:0:6670:0:(client.c:2715:ptlrpc_set_wait()) set ffff989504f1f540 going to sleep for 11 seconds 00010000:00010000:0.0:1787784913.512428:0:13825:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 9 pid: 10688 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.512446:0:13825:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 9 pid: 10688 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784913.512466:0:13825:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401835e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668468736:192.168.203.43@tcp:104: 00000100:00100000:0.0:1787784913.512494:0:13825:0:(client.c:2715:ptlrpc_set_wait()) set ffff9895063efa80 going to sleep for 11 seconds 00000100:00100000:0.0:1787784913.533283:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.533311:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.533348:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.533383:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.533427:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368000 00000100:00100000:0.0:1787784913.533437:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368000 00010000:00010000:0.0:1787784913.533506:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 39 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784913.533549:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368256 00000100:00100000:0.0:1787784913.533553:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368256 00010000:00010000:0.0:1787784913.533571:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 18 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784913.533585:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368128 00000100:00100000:0.0:1787784913.533588:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368128 00010000:00010000:0.0:1787784913.533605:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 13 pid: 22706 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787784913.533621:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368384 00000100:00100000:0.0:1787784913.533624:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368384 00000100:00100000:3.0:1787784913.533636:0:21579:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 499 00010000:00010000:0.0:1787784913.533642:0:5796:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 10 pid: 10688 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787784913.533662:0:21579:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+39:2355:x1874627658368000:12345-192.168.203.43@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787784913.533697:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 500 00010000:00010000:3.0:1787784913.533703:0:21579:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787784913.533708:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+18:2355:x1874627658368256:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787784913.533715:0:21579:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 39 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.533721:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784913.533730:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 18 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.533739:0:21579:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 39 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.533748:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 18 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.533785:0:21579:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787784913.533785:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787784913.533903:0:13825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e6b888ef/0xb0ce63137acc8ad3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b2005 expref: 18 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784913.533923:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401837480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668468352:192.168.203.43@tcp:104: 00000100:00100000:1.0:1787784913.533938:0:13825:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401835e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13825:1874627668468736:192.168.203.43@tcp:104: 00010000:00010000:1.0:1787784913.533967:0:13825:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787784913.534019:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00100000:1.0:1787784913.534029:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+13:2355:x1874627658368128:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787784913.534039:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787784913.534044:0:11303:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 13 pid: 22706 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.534055:0:11303:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 13 pid: 22706 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.534070:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784913.534076:0:11303:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787784913.534083:0:11303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.534094:0:11303:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.534114:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787784913.534204:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658368256:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 499us (890us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.534213:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 500 00000100:00100000:0.0:1787784913.534246:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 502 00000100:00100000:0.0:1787784913.534254:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063f4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+10:2355:x1874627658368384:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787784913.534263:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787784913.534268:0:5796:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 10 pid: 10688 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.534278:0:5796:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 10 pid: 10688 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.534292:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1bee9c8/0xb0ce63137acc88cd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f33 expref: 10 pid: 10688 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.534304:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784913.534307:0:5796:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787784913.534311:0:5796:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.534320:0:5796:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787784913.534340:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787784913.534493:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000023f7dba4/0xb0ce63137acc8acc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1ffe expref: 39 pid: 6670 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784913.534506:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852580 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668468480:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784913.534519:0:6670:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000092fb90ee/0xb0ce63137acc88c6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b1f2c expref: 13 pid: 22706 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787784913.534529:0:6670:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989508852d00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6670:1874627668468608:192.168.203.43@tcp:104: 00010000:00010000:2.0:1787784913.534562:0:6670:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.534573:0:6670:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.534580:0:6670:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.534591:0:6670:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.534600:0:6670:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787784913.534620:0:6670:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787784913.535578:0:21579:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658368000:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1917us (2305us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784913.535592:0:21579:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 499 00000100:00100000:1.0:1787784913.535809:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658368128:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1781us (2458us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784913.535819:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00080000:1.0:1787784913.535878:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784913.535884:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787784913.541649:0:13825:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.541678:0:13825:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.541705:0:13825:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.541715:0:13825:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787784913.541757:0:13825:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1787784913.544093:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787784913.544110:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544128:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff989537111e00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784913.544134:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544181:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544209:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544218:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d45f284/0xb0ce63137acc8bf2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6670 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544258:0:5799:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787784913.544263:0:5799:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544270:0:5799:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98953704fa00 already unused, calling callback (ffffffffc17b3420) 00010000:00010000:3.0:1787784913.544274:0:5799:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544282:0:5799:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544292:0:5799:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 1/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544307:0:5799:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050e5ecdc/0xb0ce63137acc8beb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13825 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787784913.544329:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784913.544331:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784913.544333:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784913.545567:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668468096:12345-0@lo:6:osp-syn-0-0.0 Request processed in 35128us (48769us total) trans 4294967384 rc 0/0 00000100:00100000:1.0:1787784913.545583:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000004:00080000:1.0:1787784913.546043:0:3276:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989402185e00/2, rc 0, transno 4294967384 00000100:00100000:1.0:1787784913.546052:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402185e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668468096:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1787784913.547793:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784913.548774:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063f4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658368384:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 14507us (15378us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.548793:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 502 00000100:00100000:2.0:1787784913.549486:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668468224:12345-0@lo:6:osp-syn-1-0.0 Request processed in 38692us (52140us total) trans 4294967398 rc 0/0 00000100:00100000:2.0:1787784913.549502:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 527 00000004:00080000:0.0:1787784913.549607:0:3278:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff989401834b40/2, rc 0, transno 4294967398 00000100:00100000:0.0:1787784913.549615:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668468224:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787784913.673103:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.673228:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.673251:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.673296:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.673326:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.673348:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.673377:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784913.673442:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368512 00000100:00100000:0.0:1787784913.673455:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368512 00000100:00100000:0.0:1787784913.673519:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 623 00000100:00100000:0.0:1787784913.673531:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658368512:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784913.673623:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658368512:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 94us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.673632:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 623 00000100:00100000:0.0:1787784913.673680:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368896 00000100:00100000:0.0:1787784913.673684:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368896 00000100:00100000:0.0:1787784913.673725:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369152 00000100:00100000:0.0:1787784913.673728:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369152 00000100:00100000:0.0:1787784913.673744:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368768 00000100:00100000:0.0:1787784913.673746:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368768 00000100:00100000:0.0:1787784913.673759:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369280 00000100:00100000:0.0:1787784913.673762:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369280 00000100:00100000:0.0:1787784913.673774:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 476 00000100:00100000:0.0:1787784913.673779:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658368896:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784913.673813:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658368896:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (586us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.673819:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 476 00000100:00100000:0.0:1787784913.673832:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 477 00000100:00100000:0.0:1787784913.673837:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658369152:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784913.673860:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658369152:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (609us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.673866:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 477 00000100:00100000:0.0:1787784913.673878:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 478 00000100:00100000:0.0:1787784913.673884:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527dea850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658368768:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784913.673904:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527dea850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658368768:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.673913:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 478 00000100:00100000:0.0:1787784913.674099:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 479 00000100:00100000:0.0:1787784913.674106:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658369280:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784913.674135:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658369280:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (758us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.674141:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 479 00000100:00100000:0.0:1787784913.674190:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658368640 00000100:00100000:0.0:1787784913.674194:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658368640 00000100:00100000:0.0:1787784913.674234:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369024 00000100:00100000:0.0:1787784913.674237:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369024 00000100:00100000:0.0:1787784913.674257:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 508 00000100:00100000:0.0:1787784913.674263:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658368640:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784913.674305:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658368640:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 36us (1214us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.674312:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 508 00000100:00100000:0.0:1787784913.674326:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00100000:0.0:1787784913.674332:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658369024:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784913.674359:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658369024:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (1010us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784913.674365:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00080000:0.0:1787784913.674397:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784913.842524:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1787784913.947240:0:7630:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967398 is committed 00000001:00080000:2.0:1787784914.569721:0:6696:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967384 is committed 00000040:00080000:2.0:1787784914.645804:0:23161:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784914.645819:0:23161:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784914.645874:0:23161:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 2 00000040:00080000:2.0:1787784914.645883:0:23161:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787784914.867394:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784915.443525:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784915.443567:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668468864:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784915.443641:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784915.445921:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668468864 00000100:00100000:3.0:1787784915.445935:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668468864 00000100:00100000:3.0:1787784915.446030:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1787784915.446040:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668468864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784915.446081:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784915.446192:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668468864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 154us (2550us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784915.446199:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1787784915.446242:0:3278:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967384 gen 2 00000004:00080000:3.0:1787784915.446248:0:3278:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989402185e00, transno 4294967384 00000100:00100000:3.0:1787784915.446264:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668468864:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784915.448610:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787784915.891757:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784916.913890:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784917.090344:0:23213:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784917.090357:0:23213:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784917.090407:0:23213:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 2 00000040:00080000:3.0:1787784917.096974:0:23213:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784917.489499:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668469376:0@lo:400:kworker.0 00000100:00100000:0.0:1787784917.489605:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784917.489704:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784917.489712:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668469248:0@lo:400:kworker.0 00000100:00100000:0.0:1787784917.489744:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784917.489910:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668469376 00000100:00100000:0.0:1787784917.489920:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668469376 00000100:00100000:0.0:1787784917.489993:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:0.0:1787784917.490004:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668469376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784917.490068:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668469376:12345-0@lo:400:kworker.0 Request processed in 66us (463us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784917.490076:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00080000:0.0:1787784917.490104:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784917.490140:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668469376:0@lo:400:kworker.0 00000100:00100000:2.0:1787784917.490337:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668469120:0@lo:400:kworker.0 00000100:00100000:2.0:1787784917.490374:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784917.490393:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668469632:0@lo:400:kworker.0 00000100:00100000:2.0:1787784917.490412:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784917.490430:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787784917.490447:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668468992:0@lo:400:kworker.0 00000100:00100000:2.0:1787784917.490479:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784917.490505:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894018361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668469504:0@lo:400:kworker.0 00000100:00100000:2.0:1787784917.490528:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784917.490553:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668468992 00000100:00100000:2.0:1787784917.490557:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668468992 00000100:00100000:2.0:1787784917.490586:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:2.0:1787784917.490594:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668468992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784917.490632:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668468992:12345-0@lo:400:kworker.0 Request processed in 39us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784917.490639:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:2.0:1787784917.490707:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668468992:0@lo:400:kworker.0 00000100:00100000:3.0:1787784917.491009:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668469248 00000100:00100000:3.0:1787784917.491014:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668469248 00000100:00100000:3.0:1787784917.491068:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668469120 00000100:00100000:3.0:1787784917.491071:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668469120 00000100:00100000:3.0:1787784917.491092:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668469504 00000100:00100000:3.0:1787784917.491095:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668469504 00000100:00100000:3.0:1787784917.491113:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1787784917.491121:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668469248:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784917.491185:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668469248:12345-0@lo:400:kworker.0 Request processed in 64us (1440us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784917.491191:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1787784917.491203:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:3.0:1787784917.491209:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668469120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784917.491241:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668469120:12345-0@lo:400:kworker.0 Request processed in 33us (867us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784917.491248:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00000100:00100000:3.0:1787784917.491259:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:3.0:1787784917.491266:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668469504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784917.491294:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508851a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668469504:12345-0@lo:400:kworker.0 Request processed in 29us (765us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784917.491300:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:3.0:1787784917.491333:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894018361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668469504:0@lo:400:kworker.0 00000100:00100000:3.0:1787784917.491344:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668469120:0@lo:400:kworker.0 00000100:00100000:3.0:1787784917.491370:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668469248:0@lo:400:kworker.0 00000100:00080000:1.0:1787784917.491447:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784917.491612:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668469632 00000100:00100000:0.0:1787784917.491616:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668469632 00000100:00100000:0.0:1787784917.491649:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1787784917.491655:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668469632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784917.491718:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668469632:12345-0@lo:400:kworker.0 Request processed in 63us (1304us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784917.491725:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00080000:1.0:1787784917.491894:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784917.491901:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784917.491913:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668469760:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784917.491951:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784917.492098:0:3277:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967398 gen 2 00000004:00080000:3.0:1787784917.492104:0:3277:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff989401834b40, transno 4294967398 00000100:00100000:3.0:1787784917.492124:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668469632:0@lo:400:kworker.0 00000100:00100000:0.0:1787784917.492856:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668469760 00000100:00100000:0.0:1787784917.492864:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668469760 00000100:00100000:0.0:1787784917.492909:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:0.0:1787784917.492918:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668469760:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784917.492952:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784917.493018:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668469760:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (1065us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784917.493026:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:3.0:1787784917.493193:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668469760:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1787784917.937565:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784918.792092:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784918.792186:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784918.792212:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784918.792227:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784918.792239:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784918.792254:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784918.792283:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784918.792318:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369664 00000100:00100000:2.0:1787784918.792329:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369664 00000100:00100000:1.0:1787784918.792399:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369536 00000100:00100000:2.0:1787784918.792402:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369792 00000100:00100000:1.0:1787784918.792406:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369536 00000100:00100000:2.0:1787784918.792408:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369792 00000100:00100000:1.0:1787784918.792449:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369920 00000100:00100000:1.0:1787784918.792453:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369920 00000100:00100000:2.0:1787784918.792457:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370176 00000100:00100000:2.0:1787784918.792463:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370176 00000100:00100000:1.0:1787784918.792486:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 513 00000100:00100000:2.0:1787784918.792499:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370048 00000100:00100000:1.0:1787784918.792499:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658369536:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784918.792504:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370048 00000100:00100000:2.0:1787784918.792528:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 482 00000100:00100000:2.0:1787784918.792538:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658369664:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784918.792614:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658369536:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 118us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784918.792622:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 513 00000100:00100000:1.0:1787784918.792641:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 514 00000100:00100000:1.0:1787784918.792647:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658369920:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784918.793966:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658369408 00000100:00100000:3.0:1787784918.793978:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658369408 00000100:00100000:3.0:1787784918.794028:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 625 00000100:00100000:3.0:1787784918.794037:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658369408:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784918.794198:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658369408:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 161us (2117us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784918.794207:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 625 00000100:00100000:1.0:1787784918.796791:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404008f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658369920:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 4130us (4548us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784918.796808:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 514 00000100:00100000:2.0:1787784918.796897:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658369664:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 4357us (4679us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784918.796907:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 482 00000100:00100000:2.0:1787784918.796935:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 483 00000100:00100000:2.0:1787784918.796949:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658369792:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784918.797024:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658369792:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 81us (4783us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784918.797030:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 483 00000100:00100000:2.0:1787784918.797044:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 484 00000100:00100000:2.0:1787784918.797051:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658370176:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784918.797094:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658370176:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 43us (4839us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784918.797104:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 484 00000100:00100000:2.0:1787784918.797117:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00100000:2.0:1787784918.797126:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658370048:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784918.797174:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658370048:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (4890us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784918.797181:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 485 00000100:00080000:2.0:1787784918.797224:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784918.797231:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784918.797234:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787784918.961491:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784918.987028:0:23265:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784918.987041:0:23265:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784918.987091:0:23265:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 2 00000040:00080000:0.0:1787784918.987097:0:23265:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784919.985432:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784920.577115:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784920.577173:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668469888:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784920.577273:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784920.577359:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668469888 00000100:00100000:0.0:1787784920.577368:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668469888 00000100:00100000:0.0:1787784920.577435:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:0.0:1787784920.577456:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668469888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784920.577499:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784920.577547:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668469888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784920.577555:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00080000:0.0:1787784920.577598:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784920.577652:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668469888:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1787784921.011517:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784921.144603:0:23316:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784921.144617:0:23316:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784921.144686:0:23316:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 2 00000040:00080000:3.0:1787784921.144694:0:23316:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784922.033515:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784922.291574:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784922.291587:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784922.291974:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 54 count 1 00000040:00080000:0.0:1787784922.291985:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784922.609581:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668470272:0@lo:400:kworker.0 00000100:00100000:0.0:1787784922.609690:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784922.609764:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784922.609770:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668470400:0@lo:400:kworker.0 00000100:00100000:0.0:1787784922.609802:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784922.610073:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668470144:0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610119:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784922.610172:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668470656:0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610196:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784922.610212:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787784922.610218:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668470016:0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610254:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784922.610275:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894018361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668470528:0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610294:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784922.610325:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470272 00000100:00100000:3.0:1787784922.610333:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470272 00000100:00100000:3.0:1787784922.610388:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470144 00000100:00100000:3.0:1787784922.610391:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470144 00000100:00100000:3.0:1787784922.610405:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470528 00000100:00100000:3.0:1787784922.610407:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470528 00000100:00100000:3.0:1787784922.610421:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 515 00000100:00100000:3.0:1787784922.610431:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668470272:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610484:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508853840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668470272:12345-0@lo:400:kworker.0 Request processed in 54us (794us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784922.610491:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 515 00000100:00100000:3.0:1787784922.610509:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 516 00000100:00100000:3.0:1787784922.610514:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668470144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610539:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668470144:12345-0@lo:400:kworker.0 Request processed in 26us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784922.610545:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 516 00000100:00100000:3.0:1787784922.610556:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:3.0:1787784922.610561:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668470528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610590:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6a8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668470528:12345-0@lo:400:kworker.0 Request processed in 28us (294us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784922.610596:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:3.0:1787784922.610648:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894018361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668470528:0@lo:400:kworker.0 00000100:00100000:3.0:1787784922.610660:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668470144:0@lo:400:kworker.0 00000100:00100000:2.0:1787784922.612349:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470400 00000100:00100000:2.0:1787784922.612365:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470400 00000100:00100000:2.0:1787784922.612438:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470656 00000100:00100000:2.0:1787784922.612442:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470656 00000100:00100000:2.0:1787784922.612469:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:2.0:1787784922.612480:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668470400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784922.612575:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668470400:12345-0@lo:400:kworker.0 Request processed in 97us (2772us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784922.612583:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:2.0:1787784922.612601:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:2.0:1787784922.612607:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668470656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784922.612658:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668470656:12345-0@lo:400:kworker.0 Request processed in 51us (2461us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784922.612665:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:2.0:1787784922.612725:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668470656:0@lo:400:kworker.0 00000100:00100000:0.0:1787784922.613466:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470016 00000100:00100000:0.0:1787784922.613473:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470016 00000100:00100000:0.0:1787784922.613511:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:0.0:1787784922.613518:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668470016:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1787784922.613569:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784922.613576:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668470016:12345-0@lo:400:kworker.0 Request processed in 56us (3318us total) trans 0 rc 0/0 00000100:00080000:1.0:1787784922.613579:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784922.613585:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784922.613590:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:1.0:1787784922.613612:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668470784:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784922.613659:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668470016:0@lo:400:kworker.0 00000100:00100000:1.0:1787784922.613704:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784922.613758:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668470400:0@lo:400:kworker.0 00000100:00100000:1.0:1787784922.613775:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668470272:0@lo:400:kworker.0 00000100:00100000:0.0:1787784922.614182:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470784 00000100:00100000:0.0:1787784922.614188:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470784 00000100:00100000:0.0:1787784922.614224:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:0.0:1787784922.614231:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668470784:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784922.614267:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784922.614327:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668470784:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (623us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784922.614334:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:3.0:1787784922.614474:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668470784:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1787784923.060354:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784923.413139:0:23368:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784923.413168:0:23368:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784923.419143:0:23368:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784923.419323:0:23368:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784923.912257:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784923.912297:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784923.912336:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784923.912374:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784923.912398:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784923.912423:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784923.912452:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784923.912679:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370816 00000100:00100000:1.0:1787784923.912688:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370816 00000100:00100000:1.0:1787784923.912742:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370432 00000100:00100000:1.0:1787784923.912745:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370432 00000100:00100000:1.0:1787784923.912769:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:1.0:1787784923.912778:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658370816:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784923.912858:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658370816:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 81us (521us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784923.912866:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:1.0:1787784923.912882:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00100000:1.0:1787784923.912889:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658370432:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784923.912909:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403048f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658370432:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (512us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784923.912915:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 519 00000100:00080000:1.0:1787784923.912930:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784923.913075:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370304 00000100:00100000:3.0:1787784923.913080:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370304 00000100:00100000:3.0:1787784923.913105:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 627 00000100:00100000:3.0:1787784923.913111:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658370304:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784923.913139:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658370304:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (893us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784923.913171:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 627 00000100:00100000:3.0:1787784923.913198:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370560 00000100:00100000:3.0:1787784923.913201:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370560 00000100:00100000:3.0:1787784923.913228:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371072 00000100:00100000:3.0:1787784923.913231:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371072 00000100:00100000:3.0:1787784923.913247:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370688 00000100:00100000:3.0:1787784923.913249:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370688 00000100:00100000:3.0:1787784923.913260:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658370944 00000100:00100000:3.0:1787784923.913262:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658370944 00000100:00100000:3.0:1787784923.913276:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 488 00000100:00100000:3.0:1787784923.913282:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658370560:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784923.913306:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658370560:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (1010us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784923.913312:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 488 00000100:00100000:3.0:1787784923.913323:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 489 00000100:00100000:3.0:1787784923.913328:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658371072:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784923.913349:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658371072:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (982us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784923.913354:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 489 00000100:00100000:3.0:1787784923.913364:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 490 00000100:00100000:3.0:1787784923.913383:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658370688:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784923.913406:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658370688:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (983us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784923.913411:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 490 00000100:00100000:3.0:1787784923.913422:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 491 00000100:00100000:3.0:1787784923.913428:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658370944:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784923.913450:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658370944:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (997us total) trans 0 rc 0/0 00000100:00080000:0.0:1787784923.915792:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784923.915800:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784923.916625:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 491 00010000:00010000:1.0:1787784924.081610:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784925.105495:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784925.335733:0:23421:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784925.335742:0:23421:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784925.336449:0:23421:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784925.336460:0:23421:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784925.681444:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668470912:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784925.681531:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784925.681642:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668470912 00000100:00100000:0.0:1787784925.681650:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668470912 00000100:00100000:0.0:1787784925.681723:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1787784925.681732:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668470912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784925.681766:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784925.681821:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668470912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784925.681830:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00080000:2.0:1787784925.681895:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784925.682122:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668470912:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784926.129509:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784927.158220:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784927.209031:0:23472:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784927.209771:0:23472:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784927.211282:0:23472:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784927.211298:0:23472:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784927.730656:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784927.730684:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668471040:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784927.730758:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784927.730944:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471040 00000100:00100000:0.0:1787784927.730953:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471040 00000100:00080000:2.0:1787784927.731062:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784927.731100:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:0.0:1787784927.731111:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668471040:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784927.731170:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784927.731225:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668471040:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (467us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784927.731233:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:2.0:1787784927.731311:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668471040:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784927.734793:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668471296:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.734865:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784927.734914:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787784927.734920:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668471424:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.734942:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784927.734978:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471296 00000100:00100000:1.0:1787784927.734984:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471296 00000100:00100000:1.0:1787784927.735024:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471424 00000100:00100000:1.0:1787784927.735028:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471424 00000100:00100000:1.0:1787784927.735056:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 520 00000100:00100000:1.0:1787784927.735065:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668471296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735113:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668471296:12345-0@lo:400:kworker.0 Request processed in 49us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784927.735120:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 520 00000100:00100000:1.0:1787784927.735136:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 521 00000100:00100000:1.0:1787784927.735142:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668471424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735188:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668471424:12345-0@lo:400:kworker.0 Request processed in 47us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784927.735194:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 521 00000100:00100000:1.0:1787784927.735219:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668471552:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735246:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784927.735267:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787784927.735272:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668471168:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735305:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784927.735325:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668471680:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735345:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784927.735378:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471552 00000100:00100000:1.0:1787784927.735382:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471552 00000100:00100000:1.0:1787784927.735412:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:1.0:1787784927.735418:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668471552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735451:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668471552:12345-0@lo:400:kworker.0 Request processed in 33us (203us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784927.735457:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:1.0:1787784927.735475:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471168 00000100:00100000:1.0:1787784927.735478:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471168 00000100:00100000:1.0:1787784927.735500:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 628 00000100:00100000:1.0:1787784927.735506:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668471168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735533:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668471168:12345-0@lo:400:kworker.0 Request processed in 28us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784927.735538:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 628 00000100:00080000:1.0:1787784927.735568:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784927.735572:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784927.735608:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668471168:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735621:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668471552:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735647:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668471424:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735654:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668471296:0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735684:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471680 00000100:00100000:1.0:1787784927.735687:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471680 00000100:00100000:1.0:1787784927.735713:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:1.0:1787784927.735719:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668471680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784927.735751:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668471680:12345-0@lo:400:kworker.0 Request processed in 33us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784927.735760:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000100:00080000:1.0:1787784927.735772:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784927.735787:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668471680:0@lo:400:kworker.0 00010000:00010000:3.0:1787784928.177382:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784929.033014:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784929.033060:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784929.033085:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784929.033116:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784929.033191:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784929.033222:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371200 00000100:00100000:1.0:1787784929.033231:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371200 00000100:00100000:1.0:1787784929.033284:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 629 00000100:00100000:1.0:1787784929.033293:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658371200:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784929.033408:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371328 00000100:00100000:3.0:1787784929.033413:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371328 00000100:00100000:3.0:1787784929.033454:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:3.0:1787784929.033463:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658371328:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784929.036568:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371584 00000100:00100000:0.0:1787784929.036592:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371584 00000100:00100000:0.0:1787784929.036653:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371840 00000100:00100000:0.0:1787784929.036657:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371840 00000100:00100000:0.0:1787784929.036684:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371456 00000100:00100000:0.0:1787784929.036688:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371456 00000100:00100000:0.0:1787784929.036713:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 493 00000100:00100000:0.0:1787784929.036722:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658371584:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784929.037278:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658371200:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 3981us (4156us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784929.037289:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 629 00000100:00100000:1.0:1787784929.037324:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 494 00000100:00100000:1.0:1787784929.037335:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658371840:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784929.037486:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658371328:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 4022us (4479us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784929.037496:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:3.0:1787784929.037524:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 495 00000100:00100000:3.0:1787784929.037533:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658371456:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784929.037562:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658371456:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (4371us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784929.037568:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 495 00000100:00080000:3.0:1787784929.037603:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784929.037608:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787784929.037612:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784929.037688:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658371584:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 965us (4626us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784929.037695:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 493 00000100:00100000:1.0:1787784929.037740:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658371840:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 408us (4653us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784929.037746:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 494 00000100:00100000:0.0:1787784929.038272:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784929.038389:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784929.038776:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371968 00000100:00100000:3.0:1787784929.038784:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371968 00000100:00100000:3.0:1787784929.038843:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 496 00000100:00100000:3.0:1787784929.038852:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658371968:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784929.038901:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658371968:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 50us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784929.038908:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 496 00000100:00080000:0.0:1787784929.039017:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784929.039210:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658371712 00000100:00100000:0.0:1787784929.039215:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658371712 00000100:00100000:0.0:1787784929.039256:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 524 00000100:00100000:0.0:1787784929.039263:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658371712:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784929.039295:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402aded00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658371712:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (1025us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784929.039302:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 524 00000100:00080000:3.0:1787784929.039686:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:1.0:1787784929.102834:0:23524:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784929.102844:0:23524:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784929.103395:0:23524:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784929.103411:0:23524:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784929.201347:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784930.225390:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784930.801578:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668471808:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784930.801648:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784930.801760:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471808 00000100:00100000:0.0:1787784930.801768:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471808 00000100:00100000:0.0:1787784930.801837:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:0.0:1787784930.801847:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393761c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668471808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784930.801881:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784930.801927:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393761c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668471808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784930.801934:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00080000:2.0:1787784930.802095:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784930.802139:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668471808:0@lo:13:osp-pre-0-0.0 00000040:00080000:2.0:1787784930.956942:0:23576:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784930.956950:0:23576:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784930.957922:0:23576:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784930.957934:0:23576:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784931.250385:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784932.273289:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784932.533882:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784932.533891:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784932.534449:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784932.534458:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784932.857726:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668471936:0@lo:400:kworker.0 00000040:00080000:1.0:1787784932.857730:0:23627:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784932.857736:0:23627:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:0.0:1787784932.857790:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784932.857829:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668472448:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.857854:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784932.857874:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787784932.857879:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668472320:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.857900:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784932.857927:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668472064:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.857949:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784932.857964:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668472576:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.857981:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784932.857994:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784932.857999:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668472192:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858015:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784932.858041:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668471936 00000100:00100000:0.0:1787784932.858046:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668471936 00000100:00100000:0.0:1787784932.858081:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:0.0:1787784932.858088:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668471936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858128:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668471936:12345-0@lo:400:kworker.0 Request processed in 41us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784932.858135:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:0.0:1787784932.858183:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472448 00000100:00100000:0.0:1787784932.858186:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472448 00000100:00100000:0.0:1787784932.858212:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472064 00000100:00100000:0.0:1787784932.858214:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472064 00000100:00100000:0.0:1787784932.858226:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472192 00000100:00100000:0.0:1787784932.858228:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472192 00000100:00100000:0.0:1787784932.858241:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1787784932.858246:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668472448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858274:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668472448:12345-0@lo:400:kworker.0 Request processed in 28us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784932.858280:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1787784932.858291:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:0.0:1787784932.858295:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668472064:12345-0@lo:400:kworker.0 00000040:00080000:1.0:1787784932.858319:0:23627:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000100:00100000:0.0:1787784932.858321:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b45cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668472064:12345-0@lo:400:kworker.0 Request processed in 26us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784932.858326:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000040:00080000:1.0:1787784932.858332:0:23627:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784932.858336:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 527 00000100:00100000:0.0:1787784932.858341:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668472192:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858363:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a90b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668472192:12345-0@lo:400:kworker.0 Request processed in 23us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784932.858367:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 527 00000100:00100000:0.0:1787784932.858384:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472320 00000100:00100000:0.0:1787784932.858386:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472320 00000100:00100000:0.0:1787784932.858409:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472576 00000100:00100000:0.0:1787784932.858411:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472576 00000100:00100000:0.0:1787784932.858430:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:0.0:1787784932.858436:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668472320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858465:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668472320:12345-0@lo:400:kworker.0 Request processed in 29us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784932.858470:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:0.0:1787784932.858481:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:0.0:1787784932.858486:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668472576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858510:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668472576:12345-0@lo:400:kworker.0 Request processed in 25us (528us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784932.858516:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00100000:0.0:1787784932.858572:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668472320:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858582:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668471936:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858591:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668472448:0@lo:400:kworker.0 00000100:00080000:0.0:1787784932.858615:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784932.858618:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784932.858621:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784932.858638:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668472192:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858645:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cc0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668472064:0@lo:400:kworker.0 00000100:00100000:0.0:1787784932.858653:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668472576:0@lo:400:kworker.0 00000100:00100000:1.0:1787784932.865254:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668472704:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784932.865339:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784932.865590:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472704 00000100:00100000:3.0:1787784932.865599:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472704 00000100:00100000:3.0:1787784932.865657:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1787784932.865667:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668472704:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784932.865703:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784932.865768:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668472704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 103us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784932.865775:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1787784932.865818:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668472704:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1787784933.297314:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784934.155295:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784934.155336:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784934.155360:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784934.155389:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784934.155405:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784934.155422:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784934.155477:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784934.155503:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372480 00000100:00100000:3.0:1787784934.155511:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372480 00000100:00100000:3.0:1787784934.155578:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372736 00000100:00100000:3.0:1787784934.155581:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372736 00000100:00100000:3.0:1787784934.155599:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372352 00000100:00100000:3.0:1787784934.155601:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372352 00000100:00100000:3.0:1787784934.155613:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372864 00000100:00100000:3.0:1787784934.155615:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372864 00000100:00100000:3.0:1787784934.155629:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 499 00000100:00100000:3.0:1787784934.155639:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658372480:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784934.155707:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658372480:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 69us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784934.155715:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 499 00000100:00100000:3.0:1787784934.155731:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 500 00000100:00100000:3.0:1787784934.155737:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658372736:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784934.155760:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658372736:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (400us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784934.155766:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 500 00000100:00100000:3.0:1787784934.155779:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00100000:3.0:1787784934.155784:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658372352:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784934.155807:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658372352:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784934.155813:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 501 00000100:00100000:3.0:1787784934.155823:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 502 00000100:00100000:3.0:1787784934.155830:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658372864:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784934.155850:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658372864:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784934.155857:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 502 00000100:00100000:3.0:1787784934.156208:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372224 00000100:00100000:3.0:1787784934.156213:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372224 00000100:00100000:3.0:1787784934.156244:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372608 00000100:00100000:3.0:1787784934.156246:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372608 00000100:00100000:3.0:1787784934.156263:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 528 00000100:00100000:3.0:1787784934.156270:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658372224:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784934.156299:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658372224:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (1012us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784934.156305:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 528 00000100:00100000:3.0:1787784934.156317:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 529 00000100:00100000:3.0:1787784934.156323:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658372608:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784934.156343:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658372608:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (922us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784934.156348:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 529 00000100:00080000:3.0:1787784934.156366:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784934.156371:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787784934.156373:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784934.156637:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372096 00000100:00100000:1.0:1787784934.156642:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372096 00000100:00100000:1.0:1787784934.156680:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 631 00000100:00100000:1.0:1787784934.156688:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658372096:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784934.156723:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658372096:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (1333us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784934.156729:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 631 00010000:00010000:3.0:1787784934.322728:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784934.852946:0:23678:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784934.852955:0:23678:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784934.853501:0:23678:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784934.853512:0:23678:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784935.345353:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784935.921643:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784935.921665:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a930c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668472832:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784935.921727:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784935.921852:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472832 00000100:00100000:3.0:1787784935.921860:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472832 00000100:00100000:3.0:1787784935.921928:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1787784935.921938:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668472832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784935.921971:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784935.922020:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668472832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (292us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784935.922028:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1787784935.922073:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a930c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668472832:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784935.922346:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784936.369424:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784936.760814:0:23729:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784936.760822:0:23729:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784936.761854:0:23729:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784936.761873:0:23729:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784937.396771:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784937.969957:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784937.969980:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668472960:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784937.970047:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784937.970304:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668472960 00000100:00100000:1.0:1787784937.970313:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668472960 00000100:00100000:1.0:1787784937.970379:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:1.0:1787784937.970389:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668472960:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784937.970425:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784937.970474:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668472960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 87us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784937.970480:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:1.0:1787784937.970537:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668472960:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1787784937.970603:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784937.971570:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031270c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668473472:0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.971616:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784937.971659:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787784937.971666:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668473344:0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.971698:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784937.971699:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403124f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668473216:0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.971738:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473472 00000100:00100000:2.0:1787784937.971745:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473472 00000100:00100000:3.0:1787784937.971780:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784937.971818:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1787784937.971831:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668473472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784937.971855:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787784937.971867:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895393752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668473088:0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.971894:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668473472:12345-0@lo:400:kworker.0 Request processed in 63us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784937.971902:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1787784937.971931:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473344 00000100:00100000:2.0:1787784937.971935:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473344 00000100:00100000:2.0:1787784937.971960:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473216 00000100:00100000:2.0:1787784937.971963:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473216 00000100:00100000:2.0:1787784937.971981:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1787784937.971987:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668473344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.972023:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668473344:12345-0@lo:400:kworker.0 Request processed in 36us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784937.972029:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1787784937.972040:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 531 00000100:00100000:2.0:1787784937.972047:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668473216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.972072:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668473216:12345-0@lo:400:kworker.0 Request processed in 28us (297us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784937.972078:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 531 00000100:00080000:2.0:1787784937.972102:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784937.972106:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784937.972134:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668473344:0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.972178:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031270c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668473472:0@lo:400:kworker.0 00000100:00100000:3.0:1787784937.972357:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784937.972416:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668473600:0@lo:400:kworker.0 00000100:00100000:3.0:1787784937.972461:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784937.972525:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403124f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668473216:0@lo:400:kworker.0 00000100:00100000:1.0:1787784937.972809:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473600 00000100:00100000:1.0:1787784937.972816:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473600 00000100:00100000:1.0:1787784937.972895:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 532 00000100:00100000:1.0:1787784937.972905:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668473600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784937.972984:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668473600:12345-0@lo:400:kworker.0 Request processed in 80us (522us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784937.972991:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 532 00000100:00100000:1.0:1787784937.973031:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668473600:0@lo:400:kworker.0 00000100:00080000:0.0:1787784937.973218:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784937.974661:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473088 00000100:00100000:2.0:1787784937.974676:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473088 00000100:00100000:2.0:1787784937.974749:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:2.0:1787784937.974760:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668473088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784937.974857:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668473088:12345-0@lo:400:kworker.0 Request processed in 99us (2941us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784937.974867:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:3.0:1787784937.974930:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895393752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668473088:0@lo:400:kworker.0 00010000:00010000:2.0:1787784938.417383:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784938.977272:0:23781:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784938.977285:0:23781:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784938.978887:0:23781:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784938.978919:0:23781:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784939.273899:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784939.273959:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784939.273999:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784939.274040:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784939.274060:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784939.274083:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784939.274190:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784939.274225:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658373376 00000100:00100000:1.0:1787784939.274238:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658373376 00000100:00100000:1.0:1787784939.274308:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658373632 00000100:00100000:1.0:1787784939.274311:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658373632 00000100:00100000:1.0:1787784939.274330:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658373248 00000100:00100000:1.0:1787784939.274333:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658373248 00000100:00100000:1.0:1787784939.274348:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658373760 00000100:00100000:1.0:1787784939.274350:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658373760 00000100:00100000:1.0:1787784939.274369:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 504 00000100:00100000:1.0:1787784939.274381:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658373376:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784939.274482:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658373376:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 102us (521us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784939.274491:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 504 00000100:00100000:1.0:1787784939.274522:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 505 00000100:00100000:1.0:1787784939.274529:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658373632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784939.274555:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658373632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (556us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784939.274562:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 505 00000100:00100000:1.0:1787784939.274573:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 506 00000100:00100000:1.0:1787784939.274579:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658373248:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784939.274605:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658373248:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (544us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784939.274611:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 506 00000100:00100000:1.0:1787784939.274622:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 507 00000100:00100000:1.0:1787784939.274628:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658373760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784939.274651:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658373760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (462us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784939.274657:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 507 00000100:00080000:0.0:1787784939.274910:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784939.274916:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784939.275225:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658372992 00000100:00100000:2.0:1787784939.275233:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658372992 00000100:00100000:2.0:1787784939.275282:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00100000:2.0:1787784939.275292:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658372992:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784939.275342:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658372992:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 53us (1301us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784939.275349:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00100000:2.0:1787784939.275386:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658373120 00000100:00100000:2.0:1787784939.275391:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658373120 00000100:00100000:2.0:1787784939.275428:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658373504 00000100:00100000:2.0:1787784939.275432:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658373504 00000100:00100000:2.0:1787784939.275456:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 533 00000100:00100000:2.0:1787784939.275461:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658373120:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784939.275495:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658373120:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (1627us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784939.275501:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 533 00000100:00100000:2.0:1787784939.275525:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 534 00000100:00100000:2.0:1787784939.275532:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658373504:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784939.275556:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658373504:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (1473us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784939.275563:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 534 00000100:00080000:2.0:1787784939.275602:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784939.441376:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784940.469757:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784941.008741:0:23832:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784941.008753:0:23832:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784941.009761:0:23832:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784941.009779:0:23832:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784941.058879:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784941.058916:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a930c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668473728:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784941.059016:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784941.059197:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473728 00000100:00100000:3.0:1787784941.059209:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473728 00000100:00100000:3.0:1787784941.060558:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:3.0:1787784941.060577:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668473728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784941.060628:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784941.060730:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668473728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 159us (1715us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784941.060739:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00080000:2.0:1787784941.061031:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784941.061105:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a930c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668473728:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784941.489360:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784942.513583:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784942.770324:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784942.770340:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784942.772252:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784942.772592:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:0.0:1787784942.979363:0:23883:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784942.979373:0:23883:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784942.980404:0:23883:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784942.980419:0:23883:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784943.089513:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668473856:0@lo:400:kworker.0 00000100:00100000:0.0:1787784943.089625:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784943.089738:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473856 00000100:00100000:3.0:1787784943.089748:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473856 00000100:00100000:3.0:1787784943.089812:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:3.0:1787784943.089823:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668473856:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784943.090193:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668474368:0@lo:400:kworker.0 00000100:00100000:0.0:1787784943.090232:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784943.090443:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668473856:12345-0@lo:400:kworker.0 Request processed in 620us (817us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784943.090452:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:0.0:1787784943.090614:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668473856:0@lo:400:kworker.0 00000100:00100000:0.0:1787784943.090641:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784943.090647:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668473984:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784943.090679:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784943.090734:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668474368 00000100:00100000:0.0:1787784943.090739:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668474368 00000100:00100000:0.0:1787784943.090791:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:0.0:1787784943.090799:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668474368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784943.090843:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668474368:12345-0@lo:400:kworker.0 Request processed in 44us (609us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784943.090849:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:0.0:1787784943.090890:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668474368:0@lo:400:kworker.0 00000100:00100000:3.0:1787784943.091342:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668473984 00000100:00100000:3.0:1787784943.091348:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668473984 00000100:00100000:3.0:1787784943.091386:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:3.0:1787784943.091393:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668473984:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787784943.091429:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784943.091661:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668473984:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784943.091711:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bf0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668473984:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (796us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784943.091719:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:2.0:1787784943.094697:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031243c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668474496:0@lo:400:kworker.0 00000100:00100000:2.0:1787784943.094790:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784943.094858:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403124b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668474112:0@lo:400:kworker.0 00000100:00100000:2.0:1787784943.094887:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784943.094909:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784943.094916:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668474240:0@lo:400:kworker.0 00000100:00100000:2.0:1787784943.094940:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1787784943.094969:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784943.094975:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784943.094990:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668474496 00000100:00100000:2.0:1787784943.094998:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668474496 00000100:00100000:2.0:1787784943.095055:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668474112 00000100:00100000:2.0:1787784943.095058:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668474112 00000100:00100000:2.0:1787784943.095076:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668474240 00000100:00100000:2.0:1787784943.095078:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668474240 00000100:00100000:2.0:1787784943.096044:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 535 00000100:00100000:2.0:1787784943.096058:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668474496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784943.096173:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668474496:12345-0@lo:400:kworker.0 Request processed in 119us (1383us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784943.096179:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 535 00000100:00100000:2.0:1787784943.096196:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:2.0:1787784943.096201:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668474112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784943.096227:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668474112:12345-0@lo:400:kworker.0 Request processed in 27us (1341us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784943.096231:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:2.0:1787784943.096241:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 537 00000100:00100000:2.0:1787784943.096247:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668474240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784943.096275:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668474240:12345-0@lo:400:kworker.0 Request processed in 29us (1334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784943.096281:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 537 00000100:00080000:2.0:1787784943.096439:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784943.096509:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668474240:0@lo:400:kworker.0 00000100:00100000:2.0:1787784943.096526:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403124b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668474112:0@lo:400:kworker.0 00000100:00100000:0.0:1787784943.096653:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031243c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668474496:0@lo:400:kworker.0 00010000:00010000:0.0:1787784943.540673:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784944.396676:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784944.396718:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784944.396737:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784944.396774:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784944.396796:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784944.396816:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784944.396819:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374144 00000100:00100000:0.0:1787784944.396830:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374144 00000100:00100000:1.0:1787784944.396854:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784944.396880:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374400 00000100:00100000:1.0:1787784944.396885:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374400 00000100:00100000:0.0:1787784944.396895:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374656 00000100:00100000:0.0:1787784944.396900:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374656 00000100:00100000:1.0:1787784944.396924:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374016 00000100:00100000:1.0:1787784944.396929:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374016 00000100:00100000:0.0:1787784944.396929:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374272 00000100:00100000:0.0:1787784944.396935:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374272 00000100:00100000:0.0:1787784944.396957:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374528 00000100:00100000:0.0:1787784944.396960:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374528 00000100:00100000:1.0:1787784944.396965:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 538 00000100:00100000:1.0:1787784944.396978:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658374400:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784944.396984:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00100000:0.0:1787784944.396996:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658374144:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784944.397050:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658374144:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 57us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784944.397071:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 509 00000100:00100000:0.0:1787784944.397094:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 510 00000100:00100000:3.0:1787784944.397096:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658373888 00000100:00100000:0.0:1787784944.397102:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658374656:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784944.397103:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658373888 00000100:00100000:3.0:1787784944.397445:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 635 00000100:00100000:3.0:1787784944.397454:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658373888:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:2.0:1787784944.397650:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784944.397655:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784944.397658:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784944.398451:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658374400:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1473us (1708us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784944.398461:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 538 00000100:00100000:1.0:1787784944.398492:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 539 00000100:00100000:1.0:1787784944.398501:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658374016:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784944.398547:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658374016:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 48us (1751us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784944.398553:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 539 00000100:00100000:1.0:1787784944.398574:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 511 00000100:00100000:1.0:1787784944.398581:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658374272:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784944.398604:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658374272:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (1787us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784944.398609:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 511 00000100:00100000:1.0:1787784944.398622:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 512 00000100:00100000:1.0:1787784944.398630:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658374528:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784944.398649:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658374528:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (1796us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784944.398656:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 512 00000100:00100000:0.0:1787784944.402453:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658374656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 5340us (5666us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784944.402483:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 510 00000100:00100000:3.0:1787784944.403721:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658373888:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 6261us (7047us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784944.403731:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 635 00010000:00010000:2.0:1787784944.561919:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784944.857826:0:23935:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784944.857836:0:23935:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784944.858506:0:23935:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784944.858521:0:23935:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784945.585324:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784946.165707:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784946.165729:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668474624:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784946.165798:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784946.165869:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668474624 00000100:00100000:0.0:1787784946.165876:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668474624 00000100:00100000:0.0:1787784946.165932:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00080000:1.0:1787784946.165988:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784946.166573:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668474624:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784946.170612:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784946.170709:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668474624:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4765us (4906us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784946.170719:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:3.0:1787784946.172035:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668474624:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787784946.623953:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784946.736662:0:23986:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784946.736672:0:23986:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784946.738809:0:23986:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784946.738829:0:23986:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784947.633466:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784948.210813:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668475136:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.210879:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784948.210935:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787784948.210941:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668474752:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.210971:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784948.210994:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403126940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668475264:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211019:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784948.211059:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403124b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668474880:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211082:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784948.211096:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668475392:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211111:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784948.211125:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784948.211129:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031243c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668475008:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211161:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784948.211191:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475136 00000100:00100000:2.0:1787784948.211197:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475136 00000100:00100000:2.0:1787784948.211241:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475392 00000100:00100000:2.0:1787784948.211244:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475392 00000100:00100000:2.0:1787784948.211266:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1787784948.211275:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668475136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211326:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668475136:12345-0@lo:400:kworker.0 Request processed in 51us (446us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784948.211332:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1787784948.211349:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 514 00000100:00100000:2.0:1787784948.211355:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668475392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211387:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668475392:12345-0@lo:400:kworker.0 Request processed in 33us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784948.211393:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 514 00000100:00100000:2.0:1787784948.211410:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668474752 00000100:00100000:2.0:1787784948.211414:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668474752 00000100:00100000:2.0:1787784948.211436:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:2.0:1787784948.211460:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668474752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211487:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668474752:12345-0@lo:400:kworker.0 Request processed in 29us (516us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784948.211493:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:2.0:1787784948.211519:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475264 00000100:00100000:2.0:1787784948.211523:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475264 00000100:00100000:2.0:1787784948.211544:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668474880 00000100:00100000:2.0:1787784948.211547:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668474880 00000100:00100000:2.0:1787784948.211562:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475008 00000100:00100000:2.0:1787784948.211565:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475008 00000100:00100000:2.0:1787784948.211578:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 540 00000100:00100000:2.0:1787784948.211583:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668475264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211611:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cc0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668475264:12345-0@lo:400:kworker.0 Request processed in 27us (590us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784948.211616:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 540 00000100:00100000:2.0:1787784948.211628:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 541 00000100:00100000:2.0:1787784948.211634:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668474880:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211657:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668474880:12345-0@lo:400:kworker.0 Request processed in 24us (574us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784948.211662:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 541 00000100:00100000:2.0:1787784948.211673:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 542 00000100:00100000:2.0:1787784948.211679:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668475008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211700:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668475008:12345-0@lo:400:kworker.0 Request processed in 22us (538us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784948.211705:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 542 00000100:00080000:2.0:1787784948.211729:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784948.211733:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784948.211735:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784948.211778:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031243c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668475008:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211788:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403124b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668474880:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211797:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668475392:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211823:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668474752:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211831:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403126940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668475264:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.211839:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668475136:0@lo:400:kworker.0 00000100:00100000:2.0:1787784948.215134:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668475520:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784948.215231:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784948.215873:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475520 00000100:00100000:1.0:1787784948.215883:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475520 00000100:00100000:1.0:1787784948.215938:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:1.0:1787784948.215948:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035970c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668475520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784948.215984:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784948.216045:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035970c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668475520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 99us (813us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784948.216053:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1787784948.216168:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305070c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668475520:0@lo:13:osp-pre-1-0.0 00000040:00080000:0.0:1787784948.511057:0:24037:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784948.511067:0:24037:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784948.511821:0:24037:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784948.511831:0:24037:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784948.657535:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784949.512082:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784949.512117:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784949.512179:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784949.512198:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784949.512215:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784949.512229:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784949.512249:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784949.512321:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375296 00000100:00100000:2.0:1787784949.512330:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375296 00000100:00100000:2.0:1787784949.512400:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374912 00000100:00100000:2.0:1787784949.512404:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374912 00000100:00100000:2.0:1787784949.512428:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 543 00000100:00100000:2.0:1787784949.512452:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658375296:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784949.512540:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658375296:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 103us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784949.512548:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 543 00000100:00100000:2.0:1787784949.512564:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 544 00000100:00100000:2.0:1787784949.512571:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658374912:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784949.512597:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658374912:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784949.512603:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 544 00000100:00100000:2.0:1787784949.512624:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375040 00000100:00100000:2.0:1787784949.512627:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375040 00000100:00100000:2.0:1787784949.512649:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375552 00000100:00100000:2.0:1787784949.512652:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375552 00000100:00100000:2.0:1787784949.512668:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375168 00000100:00100000:2.0:1787784949.512670:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375168 00000100:00100000:2.0:1787784949.512682:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375424 00000100:00100000:2.0:1787784949.512685:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375424 00000100:00100000:2.0:1787784949.512698:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 515 00000100:00100000:2.0:1787784949.512704:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658375040:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784949.512730:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658375040:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (613us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784949.512737:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 515 00000100:00100000:2.0:1787784949.512748:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:2.0:1787784949.512755:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658375552:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784949.512778:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658375552:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (579us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784949.512784:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 516 00000100:00100000:2.0:1787784949.512795:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:2.0:1787784949.512801:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658375168:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784949.512821:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658375168:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (591us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784949.512827:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 517 00000100:00100000:2.0:1787784949.512838:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:2.0:1787784949.512843:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658375424:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784949.512864:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658375424:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784949.512870:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 518 00000100:00100000:3.0:1787784949.513736:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658374784 00000100:00100000:3.0:1787784949.513744:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658374784 00000100:00100000:3.0:1787784949.513797:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 637 00000100:00100000:3.0:1787784949.513805:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658374784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784949.513845:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658374784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 41us (1772us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784949.513852:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 637 00000100:00080000:1.0:1787784949.513988:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784949.513994:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784949.513997:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784949.683134:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784950.402784:0:24088:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784950.402795:0:24088:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784950.405009:0:24088:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784950.405024:0:24088:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784950.705292:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784951.281429:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784951.281455:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668475648:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787784951.281524:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784951.281834:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475648 00000100:00100000:3.0:1787784951.281841:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475648 00000100:00100000:3.0:1787784951.281953:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1787784951.281964:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668475648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784951.281998:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784951.282052:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668475648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (527us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784951.282072:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00080000:1.0:1787784951.282129:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784951.282300:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668475648:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784951.729272:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784952.247246:0:24140:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784952.247255:0:24140:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784952.250008:0:24140:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784952.250034:0:24140:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784952.753339:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784953.009347:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784953.009371:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784953.010026:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784953.010037:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784953.329473:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668475776:0@lo:400:kworker.0 00000100:00100000:2.0:1787784953.329547:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784953.329600:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475776 00000100:00100000:2.0:1787784953.329607:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895393752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668476288:0@lo:400:kworker.0 00000100:00100000:3.0:1787784953.329608:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475776 00000100:00100000:2.0:1787784953.329633:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784953.329651:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 638 00000100:00100000:2.0:1787784953.329657:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784953.329661:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668475776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784953.329662:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668476160:0@lo:400:kworker.0 00000100:00100000:2.0:1787784953.329685:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784953.329755:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668475904:0@lo:400:kworker.0 00000100:00100000:2.0:1787784953.329797:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668475776:0@lo:400:kworker.0 00000100:00100000:2.0:1787784953.329827:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668476032:0@lo:400:kworker.0 00000100:00100000:1.0:1787784953.329864:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476160 00000100:00100000:1.0:1787784953.329869:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476160 00000100:00100000:1.0:1787784953.329907:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 519 00000100:00100000:1.0:1787784953.329916:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035961c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668476160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784953.332525:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668475776:12345-0@lo:400:kworker.0 Request processed in 2675us (2787us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784953.332540:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 638 00000100:00100000:3.0:1787784953.332645:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476288 00000100:00100000:3.0:1787784953.332654:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476288 00000100:00100000:3.0:1787784953.332701:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 545 00000100:00100000:3.0:1787784953.332710:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668476288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784953.332782:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668476288:12345-0@lo:400:kworker.0 Request processed in 73us (3146us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784953.332788:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 545 00000100:00100000:3.0:1787784953.332856:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895393752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668476288:0@lo:400:kworker.0 00000100:00080000:3.0:1787784953.332890:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784953.332896:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784953.334606:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784953.334662:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668476416:0@lo:400:kworker.0 00000100:00100000:0.0:1787784953.334696:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784953.334727:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784953.334734:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668476544:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784953.334757:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784953.334794:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476416 00000100:00100000:0.0:1787784953.334800:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476416 00000100:00100000:0.0:1787784953.334850:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 520 00000100:00100000:0.0:1787784953.334858:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668476416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784953.334903:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668476416:12345-0@lo:400:kworker.0 Request processed in 45us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784953.334910:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 520 00000100:00100000:0.0:1787784953.334932:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476544 00000100:00100000:0.0:1787784953.334936:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476544 00000100:00100000:0.0:1787784953.334960:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:0.0:1787784953.334966:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668476544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784953.334992:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784953.335024:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668476544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 58us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784953.335029:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00080000:0.0:1787784953.335051:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784953.335057:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668475904 00000100:00100000:3.0:1787784953.335061:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668475904 00000100:00100000:0.0:1787784953.335081:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668476544:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784953.335092:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668476416:0@lo:400:kworker.0 00000100:00100000:3.0:1787784953.335093:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:3.0:1787784953.335099:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668475904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784953.335138:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668475904:12345-0@lo:400:kworker.0 Request processed in 39us (537us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784953.335163:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:2.0:1787784953.335360:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784953.335443:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476032 00000100:00100000:3.0:1787784953.335447:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476032 00000100:00100000:3.0:1787784953.335538:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 547 00000100:00100000:3.0:1787784953.335545:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035943c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668476032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784953.335581:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035943c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668476032:12345-0@lo:400:kworker.0 Request processed in 36us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784953.335586:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 547 00000100:00100000:3.0:1787784953.335627:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668476032:0@lo:400:kworker.0 00000100:00100000:3.0:1787784953.335658:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668475904:0@lo:400:kworker.0 00000100:00100000:1.0:1787784953.336377:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035961c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668476160:12345-0@lo:400:kworker.0 Request processed in 6459us (6687us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784953.336386:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 519 00000100:00100000:3.0:1787784953.336539:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668476160:0@lo:400:kworker.0 00000100:00080000:2.0:1787784953.338051:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784953.777246:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784954.080517:0:24191:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784954.080526:0:24191:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784954.081341:0:24191:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784954.081351:0:24191:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784954.631585:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784954.631633:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784954.631667:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784954.631696:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784954.631717:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784954.631743:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784954.631767:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784954.631793:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376192 00000100:00100000:1.0:1787784954.631802:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376192 00000100:00100000:1.0:1787784954.631855:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375808 00000100:00100000:1.0:1787784954.631858:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375808 00000100:00100000:1.0:1787784954.631880:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 548 00000100:00100000:1.0:1787784954.631890:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658376192:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784954.631959:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658376192:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 71us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784954.631967:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 548 00000100:00100000:1.0:1787784954.631984:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:1.0:1787784954.631991:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658375808:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784954.632018:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658375808:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784954.632024:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:2.0:1787784954.632387:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375680 00000100:00100000:2.0:1787784954.632392:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375680 00000100:00100000:2.0:1787784954.632438:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00100000:2.0:1787784954.632446:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658375680:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784954.632479:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658375680:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (905us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784954.632485:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00080000:2.0:1787784954.632513:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784954.632582:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658375936 00000100:00100000:0.0:1787784954.632587:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658375936 00000100:00100000:0.0:1787784954.632638:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376448 00000100:00100000:0.0:1787784954.632641:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376448 00000100:00100000:0.0:1787784954.632658:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376064 00000100:00100000:0.0:1787784954.632661:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376064 00000100:00100000:0.0:1787784954.632673:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376320 00000100:00100000:0.0:1787784954.632675:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376320 00000100:00100000:0.0:1787784954.632689:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 521 00000100:00100000:0.0:1787784954.632696:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658375936:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784954.632726:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658375936:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (1101us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784954.632732:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 521 00000100:00100000:0.0:1787784954.632748:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 522 00000100:00100000:0.0:1787784954.632754:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658376448:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784954.632777:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658376448:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (1081us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784954.632783:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 522 00000100:00100000:0.0:1787784954.632794:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:0.0:1787784954.632799:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658376064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784954.632820:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658376064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (1077us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784954.632826:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 523 00000100:00100000:0.0:1787784954.632837:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 524 00000100:00100000:0.0:1787784954.632843:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658376320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784954.632862:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658376320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (1095us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784954.632869:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 524 00000100:00080000:2.0:1787784954.633079:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784954.633083:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784954.801540:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784955.828027:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784955.959258:0:24242:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784955.959271:0:24242:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784955.960057:0:24242:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784955.960081:0:24242:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784956.401739:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787784956.405657:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668476672:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784956.405835:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784956.409465:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476672 00000100:00100000:0.0:1787784956.409489:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476672 00000100:00100000:0.0:1787784956.409693:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:0.0:1787784956.409707:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668476672:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784956.409769:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784956.409902:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668476672:12345-0@lo:13:osp-pre-0-0.0 Request processed in 199us (4068us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784956.409910:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:1.0:1787784956.410188:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668476672:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1787784956.414208:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787784956.849457:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784957.861528:0:24293:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784957.861541:0:24293:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784957.862063:0:24293:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784957.862073:0:24293:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784957.873324:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784958.450549:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784958.450579:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895406652c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668476800:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784958.450675:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784958.450846:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476800 00000100:00100000:0.0:1787784958.450857:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476800 00000100:00100000:0.0:1787784958.450934:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1787784958.450945:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403597840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668476800:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784958.451003:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784958.451059:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403597840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668476800:12345-0@lo:13:osp-pre-1-0.0 Request processed in 114us (384us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784958.451067:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00080000:2.0:1787784958.451311:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784958.451525:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895406652c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668476800:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784958.454034:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784958.458393:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784958.458453:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403596580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668476928:0@lo:400:kworker.0 00000100:00100000:0.0:1787784958.458568:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784958.458648:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668477184:0@lo:400:kworker.0 00000100:00100000:3.0:1787784958.458730:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668477440:0@lo:400:kworker.0 00000100:00100000:2.0:1787784958.458871:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403594780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668477312:0@lo:400:kworker.0 00000100:00100000:0.0:1787784958.458987:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668476928 00000100:00100000:0.0:1787784958.458996:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668476928 00000100:00100000:0.0:1787784958.459066:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:0.0:1787784958.459084:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668476928:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784958.459278:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506410450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668476928:12345-0@lo:400:kworker.0 Request processed in 197us (709us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784958.459297:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00100000:0.0:1787784958.459701:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403596580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668476928:0@lo:400:kworker.0 00000100:00100000:1.0:1787784958.459969:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784958.460065:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784958.460125:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784958.460215:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477312 00000100:00100000:1.0:1787784958.460222:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477312 00000100:00080000:2.0:1787784958.460280:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784958.460283:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:1.0:1787784958.460297:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668477312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784958.460369:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668477312:12345-0@lo:400:kworker.0 Request processed in 76us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784958.460385:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00000100:00100000:2.0:1787784958.460775:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668477312:0@lo:400:kworker.0 00000100:00100000:3.0:1787784958.460826:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403595e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668477056:0@lo:400:kworker.0 00000100:00100000:3.0:1787784958.460864:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784958.460973:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477184 00000100:00100000:0.0:1787784958.461110:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477184 00000100:00080000:2.0:1787784958.461198:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784958.461256:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477440 00000100:00100000:0.0:1787784958.461261:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477440 00000100:00100000:0.0:1787784958.461306:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 550 00000100:00100000:0.0:1787784958.461315:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668477184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784958.461396:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668477184:12345-0@lo:400:kworker.0 Request processed in 82us (1427us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784958.461404:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 550 00000100:00100000:0.0:1787784958.461424:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 551 00000100:00100000:0.0:1787784958.461431:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668477440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784958.461586:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668477440:12345-0@lo:400:kworker.0 Request processed in 155us (1461us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784958.461592:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 551 00000100:00100000:0.0:1787784958.461652:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668477184:0@lo:400:kworker.0 00000100:00080000:2.0:1787784958.461724:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784958.461952:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403594000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668477440:0@lo:400:kworker.0 00000100:00100000:1.0:1787784958.462065:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477056 00000100:00100000:1.0:1787784958.462071:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477056 00000100:00100000:2.0:1787784958.462126:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 552 00000100:00100000:2.0:1787784958.462137:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668477056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784958.462291:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894011eed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668477056:12345-0@lo:400:kworker.0 Request processed in 157us (1426us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784958.462299:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 552 00000100:00100000:3.0:1787784958.462336:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403595e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668477056:0@lo:400:kworker.0 00010000:00010000:2.0:1787784958.898098:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784959.694658:0:24345:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784959.694673:0:24345:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784959.696803:0:24345:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784959.696839:0:24345:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784959.754118:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784959.754247:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784959.754285:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784959.754323:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784959.754347:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784959.754432:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376960 00000100:00100000:2.0:1787784959.754449:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376960 00000100:00100000:2.0:1787784959.754535:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377216 00000100:00100000:2.0:1787784959.754539:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377216 00000100:00100000:2.0:1787784959.754562:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376832 00000100:00100000:2.0:1787784959.754564:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376832 00000100:00100000:2.0:1787784959.754595:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 526 00000100:00100000:2.0:1787784959.754607:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658376960:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784959.754715:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658376960:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 110us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784959.754725:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 526 00000100:00100000:3.0:1787784959.754739:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784959.754756:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 527 00000100:00100000:2.0:1787784959.754765:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658377216:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784959.754793:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658377216:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (508us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784959.754801:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 527 00000100:00100000:2.0:1787784959.754813:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 528 00000100:00100000:2.0:1787784959.754819:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658376832:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784959.754843:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658376832:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784959.754849:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 528 00000100:00080000:2.0:1787784959.754870:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784959.754875:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784959.757681:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784959.757733:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376576 00000100:00100000:3.0:1787784959.757744:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376576 00000100:00100000:3.0:1787784959.757807:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 641 00000100:00100000:3.0:1787784959.757818:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658376576:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784959.757898:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658376576:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 81us (3575us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784959.757908:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 641 00000100:00100000:3.0:1787784959.757947:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377344 00000100:00100000:3.0:1787784959.757951:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377344 00000100:00100000:3.0:1787784959.758006:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 529 00000100:00100000:3.0:1787784959.758014:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658377344:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784959.758043:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658377344:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (380us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784959.758051:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 529 00000100:00100000:0.0:1787784959.759307:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658376704 00000100:00100000:0.0:1787784959.759319:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658376704 00000100:00100000:0.0:1787784959.759465:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377088 00000100:00100000:0.0:1787784959.759470:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377088 00000100:00100000:3.0:1787784959.764326:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 553 00000100:00100000:3.0:1787784959.764372:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ced2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658376704:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784959.764521:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ced2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658376704:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 177us (10420us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784959.764531:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 553 00000100:00100000:3.0:1787784959.764579:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 554 00000100:00100000:3.0:1787784959.764587:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658377088:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784959.764619:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658377088:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (9882us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784959.764627:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 554 00000100:00080000:3.0:1787784959.765313:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787784959.765322:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787784959.922658:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784960.945298:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784961.509912:0:24396:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784961.509921:0:24396:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784961.511989:0:24396:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784961.512017:0:24396:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784961.522888:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784961.522909:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668477568:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787784961.522970:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784961.523035:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477568 00000100:00100000:3.0:1787784961.523042:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477568 00000100:00100000:3.0:1787784961.523103:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1787784961.523112:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ceda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668477568:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784961.523164:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784961.523207:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ceda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668477568:12345-0@lo:13:osp-pre-0-0.0 Request processed in 96us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784961.523215:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00080000:3.0:1787784961.523235:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784961.523267:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668477568:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784961.969379:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787784962.999752:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784963.250365:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784963.250377:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784963.251087:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784963.251101:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1787784963.354938:0:24447:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784963.354950:0:24447:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784963.355646:0:24447:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784963.355660:0:24447:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784963.569436:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668477696:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.569544:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784963.569559:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668477824:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784963.569603:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784963.569661:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895407392c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668478208:0@lo:400:kworker.0 00000100:00100000:2.0:1787784963.569694:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668478336:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.569701:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784963.569737:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784963.569752:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668477952:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.569784:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784963.569787:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477824 00000100:00100000:2.0:1787784963.569797:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477824 00000100:00100000:0.0:1787784963.569807:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668478464:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.569831:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784963.569855:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787784963.569861:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668478080:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.569887:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784963.569921:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:0.0:1787784963.569928:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478208 00000100:00100000:0.0:1787784963.569933:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478208 00000100:00100000:2.0:1787784963.569934:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668477824:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784963.569973:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478464 00000100:00100000:0.0:1787784963.569977:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478464 00002000:00000024:2.0:1787784963.569984:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784963.570037:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668477824:12345-0@lo:13:osp-pre-1-0.0 Request processed in 107us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784963.570045:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:2.0:1787784963.570072:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478336 00000100:00100000:2.0:1787784963.570076:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478336 00000100:00100000:2.0:1787784963.570105:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477952 00000100:00100000:2.0:1787784963.570108:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477952 00000100:00100000:2.0:1787784963.570125:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478080 00000100:00100000:2.0:1787784963.570128:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478080 00000100:00100000:2.0:1787784963.570179:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 555 00000100:00100000:2.0:1787784963.570186:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668478336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784963.570272:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668477696 00000100:00100000:1.0:1787784963.570278:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668477696 00000100:00100000:2.0:1787784963.570302:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668478336:12345-0@lo:400:kworker.0 Request processed in 116us (565us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784963.570311:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 555 00000100:00100000:2.0:1787784963.570326:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 556 00000100:00100000:2.0:1787784963.570332:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668477952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784963.570517:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402184b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668477952:12345-0@lo:400:kworker.0 Request processed in 184us (733us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784963.570524:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 556 00000100:00100000:2.0:1787784963.570537:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 557 00000100:00100000:2.0:1787784963.570543:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668478080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784963.570571:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668478080:12345-0@lo:400:kworker.0 Request processed in 28us (684us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784963.570577:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 557 00000100:00080000:2.0:1787784963.570596:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784963.570601:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784963.570602:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784963.570667:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668477824:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784963.570681:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668478336:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.571264:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:3.0:1787784963.571272:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 531 00000100:00100000:0.0:1787784963.571280:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668478208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784963.571282:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668478464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784963.571354:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668478464:12345-0@lo:400:kworker.0 Request processed in 74us (1522us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784963.571358:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668478208:12345-0@lo:400:kworker.0 Request processed in 85us (1660us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784963.571361:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 531 00000100:00100000:0.0:1787784963.571364:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1787784963.571493:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895407392c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668478208:0@lo:400:kworker.0 00000100:00100000:1.0:1787784963.571508:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:1.0:1787784963.571520:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668477696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784963.571557:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506411c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668477696:12345-0@lo:400:kworker.0 Request processed in 43us (2016us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784963.571562:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:2.0:1787784963.571588:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668477696:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.571762:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392530c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668478080:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.571777:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895392521c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668477952:0@lo:400:kworker.0 00000100:00100000:0.0:1787784963.571788:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668478464:0@lo:400:kworker.0 00010000:00010000:2.0:1787784964.019677:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784964.874792:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784964.874853:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784964.875123:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377472 00000100:00100000:0.0:1787784964.875170:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377472 00000100:00100000:0.0:1787784964.875238:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:0.0:1787784964.875251:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658377472:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784964.875521:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658377472:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 271us (742us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784964.875531:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:2.0:1787784964.877920:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377600 00000100:00100000:2.0:1787784964.877935:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377600 00000100:00100000:2.0:1787784964.878017:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 558 00000100:00100000:2.0:1787784964.878032:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658377600:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784964.878216:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cecf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658377600:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 108us (3281us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784964.878228:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 558 00000100:00080000:2.0:1787784964.878254:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784964.878552:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784964.878613:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377856 00000100:00100000:3.0:1787784964.878619:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377856 00000100:00100000:3.0:1787784964.878668:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 532 00000100:00100000:3.0:1787784964.878677:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658377856:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784964.878703:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784964.878727:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658377856:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 51us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784964.878802:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 532 00000100:00100000:3.0:1787784964.878816:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658378112 00000100:00100000:3.0:1787784964.878820:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658378112 00000100:00100000:3.0:1787784964.878847:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 533 00000100:00100000:3.0:1787784964.878854:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658378112:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784964.879071:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784964.879102:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784964.879133:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:2.0:1787784964.879242:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784964.879246:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784964.879257:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377728 00000100:00100000:2.0:1787784964.879261:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377728 00000100:00100000:2.0:1787784964.879286:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658378240 00000100:00100000:2.0:1787784964.879289:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658378240 00000100:00100000:2.0:1787784964.879317:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 534 00000100:00100000:2.0:1787784964.879325:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658377728:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784964.879369:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658377728:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 45us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784964.879537:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 534 00000100:00100000:2.0:1787784964.879553:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 535 00000100:00100000:2.0:1787784964.879560:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658378240:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784964.879588:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658378240:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (455us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784964.879595:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 535 00000100:00100000:2.0:1787784964.879621:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658377984 00000100:00100000:2.0:1787784964.879624:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658377984 00000100:00100000:2.0:1787784964.879659:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 559 00000100:00100000:2.0:1787784964.879666:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658377984:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784964.879707:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392521c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658377984:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (604us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784964.879715:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 559 00000100:00080000:2.0:1787784964.879959:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784964.879963:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784964.879966:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784964.881602:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658378112:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 2732us (2885us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784964.881620:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 533 00010000:00010000:2.0:1787784965.044768:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784965.296977:0:24498:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784965.296992:0:24498:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784965.299120:0:24498:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784965.299177:0:24498:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784966.066214:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784966.644527:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668478592:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784966.644596:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784966.644802:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478592 00000100:00100000:1.0:1787784966.644810:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478592 00000100:00100000:1.0:1787784966.644883:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:1.0:1787784966.644894:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668478592:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1787784966.645029:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00000024:1.0:1787784966.645201:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784966.645255:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668478592:12345-0@lo:13:osp-pre-0-0.0 Request processed in 363us (658us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784966.645262:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:0.0:1787784966.645410:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668478592:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784967.089524:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784967.268773:0:24550:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784967.268786:0:24550:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784967.272534:0:24550:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784967.272561:0:24550:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784968.114257:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784968.689524:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787784968.689548:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668478720:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784968.689617:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784968.689679:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478720 00000100:00100000:1.0:1787784968.689686:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478720 00000100:00100000:1.0:1787784968.689739:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:1.0:1787784968.689749:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668478720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784968.689783:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784968.689826:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668478720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784968.689833:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00080000:1.0:1787784968.689852:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787784968.689887:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668478720:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784968.691696:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668478976:0@lo:400:kworker.0 00000100:00100000:2.0:1787784968.691761:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784968.691839:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787784968.691845:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668479104:0@lo:400:kworker.0 00000100:00100000:2.0:1787784968.691869:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784968.691900:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478976 00000100:00100000:2.0:1787784968.691906:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478976 00000100:00100000:2.0:1787784968.691943:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668479104 00000100:00100000:2.0:1787784968.691946:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668479104 00000100:00100000:2.0:1787784968.691971:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 560 00000100:00100000:2.0:1787784968.691980:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668478976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784968.692025:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895407392c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668478976:12345-0@lo:400:kworker.0 Request processed in 46us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784968.692032:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 560 00000100:00100000:2.0:1787784968.692050:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 561 00000100:00100000:2.0:1787784968.692057:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668479104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784968.692082:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668479104:12345-0@lo:400:kworker.0 Request processed in 26us (212us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784968.692087:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 561 00000100:00100000:2.0:1787784968.692133:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668479104:0@lo:400:kworker.0 00000100:00100000:2.0:1787784968.692192:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540738f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668478976:0@lo:400:kworker.0 00000100:00100000:0.0:1787784968.692334:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668479232:0@lo:400:kworker.0 00000100:00100000:0.0:1787784968.692369:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784968.692402:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1787784968.692407:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539253480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668478848:0@lo:400:kworker.0 00000100:00100000:0.0:1787784968.692436:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784968.692455:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668479360:0@lo:400:kworker.0 00000100:00100000:0.0:1787784968.692476:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784968.692503:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668479360 00000100:00100000:0.0:1787784968.692507:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668479360 00000100:00100000:0.0:1787784968.692534:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00100000:0.0:1787784968.692541:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668479360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784968.692577:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668479360:12345-0@lo:400:kworker.0 Request processed in 37us (99us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784968.692584:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00100000:0.0:1787784968.692615:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668479360:0@lo:400:kworker.0 00000100:00100000:2.0:1787784968.692857:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668479232 00000100:00100000:2.0:1787784968.692862:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668479232 00000100:00100000:2.0:1787784968.692904:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:2.0:1787784968.692911:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668479232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784968.692947:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668479232:12345-0@lo:400:kworker.0 Request processed in 36us (578us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784968.692953:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:0.0:1787784968.693021:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668479232:0@lo:400:kworker.0 00000100:00100000:1.0:1787784968.693092:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668478848 00000100:00100000:1.0:1787784968.693096:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668478848 00000100:00100000:1.0:1787784968.693133:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:1.0:1787784968.693139:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668478848:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784968.693250:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668478848:12345-0@lo:400:kworker.0 Request processed in 110us (812us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784968.693256:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00080000:1.0:1787784968.693283:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784968.693287:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784968.698986:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539253480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668478848:0@lo:400:kworker.0 00010000:00010000:3.0:1787784969.141136:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784969.186772:0:24601:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784969.186787:0:24601:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784969.187884:0:24601:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784969.187903:0:24601:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784969.999725:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784969.999791:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784969.999822:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784969.999865:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784969.999908:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784970.000047:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784970.000076:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784970.000114:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658378368 00000100:00100000:2.0:1787784970.000126:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658378368 00000100:00100000:2.0:1787784970.000239:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 645 00000100:00100000:2.0:1787784970.000252:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658378368:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784970.000258:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658378496 00000100:00100000:0.0:1787784970.000265:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658378496 00000100:00100000:0.0:1787784970.000327:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658378880 00000100:00100000:0.0:1787784970.000331:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658378880 00000100:00100000:0.0:1787784970.000360:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:0.0:1787784970.000368:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658378496:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784970.000409:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658378496:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (704us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784970.000417:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:0.0:1787784970.000445:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 564 00000100:00100000:0.0:1787784970.000451:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658378880:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784970.000477:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658378880:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (431us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784970.000484:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 564 00000100:00080000:0.0:1787784970.000504:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784970.000551:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658378752 00000100:00100000:0.0:1787784970.000555:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658378752 00000100:00100000:3.0:1787784970.000585:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379008 00000100:00100000:0.0:1787784970.000591:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658378624 00000100:00100000:3.0:1787784970.000592:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379008 00000100:00100000:0.0:1787784970.000596:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658378624 00000100:00100000:3.0:1787784970.000636:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379136 00000100:00100000:3.0:1787784970.000640:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379136 00000100:00100000:2.0:1787784970.000679:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658378368:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 429us (812us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784970.000689:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 645 00000100:00100000:0.0:1787784970.000709:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 537 00000100:00100000:0.0:1787784970.000720:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658378752:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784970.000736:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 538 00000100:00100000:2.0:1787784970.000746:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658379008:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784970.000762:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658378752:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (963us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784970.000771:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 537 00000100:00100000:2.0:1787784970.000784:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658379008:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (958us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784970.000791:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 539 00000100:00100000:0.0:1787784970.000800:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658378624:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784970.000824:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658378624:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (917us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784970.008865:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 538 00000100:00100000:0.0:1787784970.008890:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 539 00000100:00100000:0.0:1787784970.008930:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 540 00000100:00100000:0.0:1787784970.008954:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658379136:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784970.009059:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658379136:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 119us (8982us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784970.009068:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 540 00000100:00080000:0.0:1787784970.009098:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784970.009641:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787784970.161425:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784970.930127:0:24652:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784970.930138:0:24652:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784970.930957:0:24652:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784970.930968:0:24652:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784971.185312:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784971.761528:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668479488:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784971.761602:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784971.761855:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668479488 00000100:00100000:1.0:1787784971.761864:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668479488 00000100:00100000:1.0:1787784971.761934:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:1.0:1787784971.761945:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668479488:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784971.761979:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784971.762037:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668479488:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784971.762043:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:3.0:1787784971.762135:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668479488:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784971.762263:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787784972.216823:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784972.767683:0:24703:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784972.767693:0:24703:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784972.768679:0:24703:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784972.768694:0:24703:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784973.233696:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784973.489400:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784973.489409:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784973.490434:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784973.490447:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784973.809435:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668479616:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784973.809518:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784973.809807:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668480000:0@lo:400:kworker.0 00000100:00100000:3.0:1787784973.809840:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784973.809864:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784973.809870:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668480128:0@lo:400:kworker.0 00000100:00100000:3.0:1787784973.809894:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784973.810029:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668479744:0@lo:400:kworker.0 00000100:00100000:0.0:1787784973.810071:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784973.810111:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668480256:0@lo:400:kworker.0 00000100:00100000:0.0:1787784973.810133:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784973.810175:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784973.810181:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668479872:0@lo:400:kworker.0 00000100:00100000:0.0:1787784973.810205:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784973.810236:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480000 00000100:00100000:0.0:1787784973.810243:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480000 00000100:00100000:0.0:1787784973.810311:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480256 00000100:00100000:0.0:1787784973.810315:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480256 00000100:00100000:0.0:1787784973.810334:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668479872 00000100:00100000:0.0:1787784973.810337:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668479872 00000100:00100000:0.0:1787784973.810354:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 565 00000100:00100000:0.0:1787784973.810362:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ced680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668480000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784973.810411:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ced680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668480000:12345-0@lo:400:kworker.0 Request processed in 50us (570us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784973.810418:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 565 00000100:00100000:0.0:1787784973.810441:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 566 00000100:00100000:0.0:1787784973.810446:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668480256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784973.810479:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668480256:12345-0@lo:400:kworker.0 Request processed in 33us (345us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784973.810486:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 566 00000100:00100000:0.0:1787784973.810499:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 567 00000100:00100000:0.0:1787784973.810504:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668479872:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784973.810526:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668479872:12345-0@lo:400:kworker.0 Request processed in 23us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784973.810532:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 567 00000100:00080000:0.0:1787784973.810556:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784973.810588:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668479872:0@lo:400:kworker.0 00000100:00100000:0.0:1787784973.810598:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668480256:0@lo:400:kworker.0 00000100:00100000:2.0:1787784973.810841:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480128 00000100:00100000:2.0:1787784973.810846:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480128 00000100:00100000:2.0:1787784973.810891:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 541 00000100:00100000:2.0:1787784973.810898:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668480128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784973.810934:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668480128:12345-0@lo:400:kworker.0 Request processed in 38us (1040us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784973.810943:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 541 00000100:00100000:3.0:1787784973.811005:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668479744 00000100:00100000:3.0:1787784973.811009:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668479744 00000100:00100000:3.0:1787784973.811039:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:3.0:1787784973.811045:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668479744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784973.811139:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668479744:12345-0@lo:400:kworker.0 Request processed in 93us (1067us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784973.811164:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:3.0:1787784973.811199:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668480128:0@lo:400:kworker.0 00000100:00100000:3.0:1787784973.811207:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668480000:0@lo:400:kworker.0 00000100:00100000:3.0:1787784973.811226:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668479744:0@lo:400:kworker.0 00000100:00100000:1.0:1787784973.811358:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668479616 00000100:00100000:1.0:1787784973.811363:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668479616 00000100:00100000:1.0:1787784973.811401:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:1.0:1787784973.811408:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668479616:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784973.811435:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784973.811484:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668479616:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (1966us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784973.811490:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00080000:0.0:1787784973.811579:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784973.811583:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787784973.811641:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668479616:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1787784974.257698:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784974.750111:0:24755:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784974.750361:0:24755:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784974.750944:0:24755:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784974.750956:0:24755:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784975.112534:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784975.112580:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784975.112685:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379392 00000100:00100000:0.0:1787784975.112694:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379392 00000100:00100000:0.0:1787784975.112761:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 568 00000100:00100000:0.0:1787784975.112771:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403126940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658379392:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784975.112837:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403126940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658379392:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 69us (313us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784975.112845:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 568 00000100:00100000:2.0:1787784975.113855:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00080000:3.0:1787784975.114088:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784975.114634:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784975.114662:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784975.114695:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379264 00000100:00100000:3.0:1787784975.114701:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379264 00000100:00100000:3.0:1787784975.114736:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 647 00000100:00100000:3.0:1787784975.114744:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658379264:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784975.114798:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658379264:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 54us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784975.114805:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 647 00000100:00100000:1.0:1787784975.116697:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379648 00000100:00100000:1.0:1787784975.116710:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379648 00000100:00100000:1.0:1787784975.116782:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379904 00000100:00100000:1.0:1787784975.116786:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379904 00000100:00100000:1.0:1787784975.116814:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 542 00000100:00100000:1.0:1787784975.116824:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658379648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784975.116903:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658379648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 81us (4322us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784975.116910:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 542 00000100:00100000:1.0:1787784975.116925:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 543 00000100:00100000:1.0:1787784975.116933:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658379904:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784975.116956:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658379904:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (3107us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784975.116962:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 543 00000100:00080000:1.0:1787784975.117847:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784975.117853:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784975.118911:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784975.118970:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784975.118997:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379520 00000100:00100000:2.0:1787784975.119005:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379520 00000100:00100000:2.0:1787784975.119060:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380032 00000100:00100000:2.0:1787784975.119063:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380032 00000100:00100000:2.0:1787784975.119084:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 544 00000100:00100000:2.0:1787784975.119092:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658379520:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784975.119164:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658379520:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 56us (4484us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784975.119172:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 544 00000100:00100000:2.0:1787784975.119186:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 545 00000100:00100000:2.0:1787784975.119192:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658380032:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784975.119216:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658380032:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784975.119224:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 545 00000100:00080000:2.0:1787784975.119238:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784975.119241:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784975.119588:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658379776 00000100:00100000:1.0:1787784975.119593:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658379776 00000100:00100000:1.0:1787784975.119638:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 569 00000100:00100000:1.0:1787784975.119645:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658379776:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784975.119677:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031243c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658379776:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (771us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784975.119684:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 569 00000100:00080000:2.0:1787784975.120014:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787784975.281854:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784976.316305:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784976.738217:0:24806:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784976.738229:0:24806:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784976.738886:0:24806:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784976.738899:0:24806:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784976.883070:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784976.883096:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668480384:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784976.883189:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784976.883238:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480384 00000100:00100000:1.0:1787784976.883255:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480384 00000100:00100000:1.0:1787784976.883321:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:1.0:1787784976.883331:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668480384:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784976.883375:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00080000:0.0:1787784976.883384:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784976.883430:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668480384:12345-0@lo:13:osp-pre-0-0.0 Request processed in 100us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784976.883437:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:0.0:1787784976.883481:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668480384:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784977.329322:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784978.356744:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787784978.645559:0:24857:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787784978.645570:0:24857:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787784978.646616:0:24857:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787784978.646637:0:24857:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787784978.929394:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668480768:0@lo:400:kworker.0 00000100:00100000:3.0:1787784978.929466:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784978.929521:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668480640:0@lo:400:kworker.0 00000100:00100000:3.0:1787784978.929546:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787784978.929552:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668480896:0@lo:400:kworker.0 00000100:00100000:2.0:1787784978.929552:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784978.929573:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668481152:0@lo:400:kworker.0 00000100:00100000:3.0:1787784978.929578:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784978.929595:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784978.929611:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787784978.929614:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480768 00000100:00100000:2.0:1787784978.929615:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668480512:0@lo:400:kworker.0 00000100:00100000:3.0:1787784978.929620:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480768 00000100:00100000:2.0:1787784978.929648:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784978.929665:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668481024:0@lo:400:kworker.0 00000100:00100000:3.0:1787784978.929679:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480640 00000100:00100000:3.0:1787784978.929682:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480640 00000100:00100000:2.0:1787784978.929696:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784978.929702:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481024 00000100:00100000:3.0:1787784978.929706:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481024 00000100:00100000:3.0:1787784978.929722:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 570 00000100:00100000:3.0:1787784978.929730:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668480768:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1787784978.929859:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784978.929890:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668480768:12345-0@lo:400:kworker.0 Request processed in 160us (423us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784978.930866:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 571 00000100:00100000:2.0:1787784978.930882:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668480640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784978.930938:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668480640:12345-0@lo:400:kworker.0 Request processed in 62us (1384us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784978.930946:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 571 00000100:00100000:2.0:1787784978.930971:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 572 00000100:00100000:2.0:1787784978.930978:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668481024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784978.931008:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668481024:12345-0@lo:400:kworker.0 Request processed in 31us (1311us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784978.931014:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 572 00000100:00100000:2.0:1787784978.931038:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480896 00000100:00100000:2.0:1787784978.931042:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480896 00000100:00100000:2.0:1787784978.931077:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481152 00000100:00100000:2.0:1787784978.931080:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481152 00000100:00100000:2.0:1787784978.931104:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:2.0:1787784978.931110:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668480896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784978.931163:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668480896:12345-0@lo:400:kworker.0 Request processed in 37us (1568us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784978.931169:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:2.0:1787784978.931182:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 547 00000100:00100000:2.0:1787784978.931188:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668481152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784978.931223:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668481152:12345-0@lo:400:kworker.0 Request processed in 36us (1627us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784978.931237:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 547 00000100:00100000:2.0:1787784978.931450:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668480512 00000100:00100000:2.0:1787784978.931455:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668480512 00000100:00100000:2.0:1787784978.931757:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 648 00000100:00100000:2.0:1787784978.931766:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668480512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784978.931803:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668480512:12345-0@lo:400:kworker.0 Request processed in 39us (2155us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784978.931809:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 648 00000100:00100000:2.0:1787784978.931844:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784978.931858:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668481280:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784978.931888:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784978.931922:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481280 00000100:00100000:2.0:1787784978.932258:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481280 00000100:00100000:2.0:1787784978.932289:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1787784978.932295:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668481280:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784978.932752:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784978.932820:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668481280:12345-0@lo:13:osp-pre-1-0.0 Request processed in 524us (931us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784978.932827:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00080000:0.0:1787784978.932955:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784978.932961:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784978.932996:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668480512:0@lo:400:kworker.0 00000100:00100000:0.0:1787784978.933012:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668481024:0@lo:400:kworker.0 00000100:00100000:0.0:1787784978.933025:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668480640:0@lo:400:kworker.0 00000100:00100000:0.0:1787784978.933036:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668481152:0@lo:400:kworker.0 00000100:00100000:3.0:1787784978.933067:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 570 00000100:00100000:0.0:1787784978.933091:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668480896:0@lo:400:kworker.0 00000100:00100000:0.0:1787784978.933103:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668480768:0@lo:400:kworker.0 00000100:00100000:3.0:1787784978.933113:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895406661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668481280:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1787784979.377378:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784980.232041:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784980.232142:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784980.232239:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784980.232268:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784980.232290:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784980.232317:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784980.232381:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380544 00000100:00100000:3.0:1787784980.232393:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380544 00000100:00100000:3.0:1787784980.232465:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380800 00000100:00100000:3.0:1787784980.232468:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380800 00000100:00100000:3.0:1787784980.232492:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380416 00000100:00100000:3.0:1787784980.232495:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380416 00000100:00100000:3.0:1787784980.232521:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 548 00000100:00100000:3.0:1787784980.232531:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658380544:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784980.232624:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658380544:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 94us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784980.232632:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 548 00000100:00100000:3.0:1787784980.232653:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:3.0:1787784980.232659:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658380800:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784980.232685:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658380800:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784980.232697:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 549 00000100:00100000:3.0:1787784980.232709:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 550 00000100:00100000:3.0:1787784980.232715:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658380416:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784980.232737:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658380416:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (447us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784980.232744:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 550 00000100:00100000:3.0:1787784980.232765:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380160 00000100:00100000:3.0:1787784980.232769:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380160 00000100:00100000:3.0:1787784980.232796:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00100000:3.0:1787784980.232803:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658380160:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784980.232828:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658380160:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (559us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784980.232834:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00080000:0.0:1787784980.233570:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787784980.233579:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787784980.235462:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380288 00000100:00100000:2.0:1787784980.235482:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380288 00000100:00080000:3.0:1787784980.235579:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784980.235628:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787784980.235707:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380928 00000100:00100000:3.0:1787784980.235711:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380928 00000100:00100000:3.0:1787784980.235770:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 551 00000100:00100000:3.0:1787784980.235783:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658380928:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784980.235879:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658380928:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 98us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784980.235888:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 551 00000100:00080000:1.0:1787784980.236381:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787784980.237206:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658380672 00000100:00100000:0.0:1787784980.237215:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658380672 00000100:00100000:0.0:1787784980.237293:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:0.0:1787784980.237304:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658380288:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784980.237379:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658380288:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (5353us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784980.237386:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:0.0:1787784980.237404:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 574 00000100:00100000:0.0:1787784980.237410:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658380672:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784980.237435:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658380672:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (5118us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784980.237442:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 574 00000100:00080000:1.0:1787784980.237831:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787784980.402548:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784980.410104:0:24908:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784980.410113:0:24908:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784980.411752:0:24908:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784980.411883:0:24908:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787784981.430278:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784982.001387:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784982.001411:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668481408:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784982.001479:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784982.001774:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481408 00000100:00100000:0.0:1787784982.001781:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481408 00000100:00100000:0.0:1787784982.001842:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1787784982.001852:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668481408:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787784982.001884:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784982.001928:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668481408:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (448us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784982.001935:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1787784982.001983:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668481408:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787784982.002036:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:2.0:1787784982.148762:0:24960:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784982.148773:0:24960:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784982.153511:0:24960:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784982.153526:0:24960:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784982.449755:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787784983.473320:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784983.730751:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784983.730761:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784983.731341:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784983.731350:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:2.0:1787784983.914621:0:25011:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784983.914630:0:25011:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784983.915415:0:25011:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784983.915490:0:25011:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787784984.049323:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784984.049355:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668481664:0@lo:400:kworker.0 00000100:00100000:2.0:1787784984.049376:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668481792:0@lo:400:kworker.0 00000100:00100000:3.0:1787784984.049419:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668482048:0@lo:400:kworker.0 00000100:00100000:3.0:1787784984.052854:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784984.052929:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787784984.052940:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668482176:0@lo:400:kworker.0 00000100:00100000:2.0:1787784984.052944:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784984.052969:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784984.052969:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668482304:0@lo:400:kworker.0 00000100:00100000:2.0:1787784984.052990:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784984.052999:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668481920:0@lo:400:kworker.0 00000100:00100000:3.0:1787784984.053021:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784984.053022:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482048 00000100:00100000:2.0:1787784984.053029:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482048 00000100:00100000:3.0:1787784984.053046:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481792 00000100:00100000:3.0:1787784984.053050:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481792 00000100:00100000:3.0:1787784984.053081:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482176 00000100:00100000:2.0:1787784984.053081:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482304 00000100:00100000:3.0:1787784984.053084:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482176 00000100:00100000:2.0:1787784984.053084:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482304 00000100:00100000:3.0:1787784984.053099:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481920 00000100:00100000:3.0:1787784984.053101:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481920 00000100:00100000:2.0:1787784984.053102:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 552 00000100:00100000:2.0:1787784984.053109:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668482048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784984.053113:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:3.0:1787784984.053118:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668481792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787784984.053178:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508852580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668481792:12345-0@lo:400:kworker.0 Request processed in 60us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784984.053186:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:3.0:1787784984.053212:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 576 00000100:00100000:3.0:1787784984.053218:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668482176:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784984.053456:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 553 00000100:00100000:0.0:1787784984.053466:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668482304:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784984.053501:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404009a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668482304:12345-0@lo:400:kworker.0 Request processed in 38us (510us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784984.053507:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 553 00000100:00100000:1.0:1787784984.053650:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784984.053686:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668481536:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784984.053712:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784984.053885:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481664 00000100:00100000:0.0:1787784984.053889:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481664 00000100:00100000:0.0:1787784984.053926:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:0.0:1787784984.053933:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668481664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784984.053971:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668481664:12345-0@lo:400:kworker.0 Request processed in 38us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784984.053977:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:2.0:1787784984.053979:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668482048:12345-0@lo:400:kworker.0 Request processed in 205us (467us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784984.053985:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 552 00000100:00100000:0.0:1787784984.054006:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668481536 00000100:00100000:0.0:1787784984.054009:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668481536 00000100:00100000:0.0:1787784984.054027:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:0.0:1787784984.054034:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668481536:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787784984.054059:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787784984.054090:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668481536:12345-0@lo:13:osp-pre-1-0.0 Request processed in 57us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784984.054096:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:0.0:1787784984.054125:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668481664:0@lo:400:kworker.0 00000100:00100000:0.0:1787784984.054144:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668481536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784984.054610:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668482176:12345-0@lo:400:kworker.0 Request processed in 32us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784984.054618:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 576 00000100:00100000:2.0:1787784984.054728:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 577 00000100:00100000:2.0:1787784984.054738:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668481920:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784984.055063:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668481920:12345-0@lo:400:kworker.0 Request processed in 327us (2040us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784984.055070:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 577 00000100:00080000:2.0:1787784984.055087:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784984.055092:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784984.055095:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784984.055230:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668481792:0@lo:400:kworker.0 00000100:00100000:1.0:1787784984.055244:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668482304:0@lo:400:kworker.0 00000100:00100000:0.0:1787784984.055381:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668481920:0@lo:400:kworker.0 00000100:00100000:0.0:1787784984.056326:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668482176:0@lo:400:kworker.0 00000100:00100000:0.0:1787784984.056341:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668482048:0@lo:400:kworker.0 00010000:00010000:0.0:1787784984.499765:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784985.365281:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784985.365320:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784985.365345:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784985.365366:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784985.365383:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784985.365400:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784985.365563:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381056 00000100:00100000:2.0:1787784985.365572:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381056 00000100:00100000:2.0:1787784985.365618:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 651 00000100:00100000:2.0:1787784985.365627:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658381056:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784985.365788:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381184 00000100:00100000:1.0:1787784985.365793:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381184 00000100:00100000:1.0:1787784985.365828:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381568 00000100:00100000:1.0:1787784985.365831:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381568 00000100:00100000:1.0:1787784985.365854:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 578 00000100:00100000:1.0:1787784985.365861:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658381184:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784985.369769:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658381056:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 3776us (4130us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784985.369790:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 651 00000100:00100000:0.0:1787784985.370195:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784985.370246:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381440 00000100:00100000:0.0:1787784985.370254:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381440 00000100:00100000:0.0:1787784985.370311:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381696 00000100:00100000:0.0:1787784985.370314:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381696 00000100:00100000:0.0:1787784985.370333:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381312 00000100:00100000:0.0:1787784985.370335:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381312 00000100:00100000:0.0:1787784985.370349:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381824 00000100:00100000:0.0:1787784985.370352:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381824 00000100:00100000:0.0:1787784985.370367:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 554 00000100:00100000:0.0:1787784985.370374:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658381440:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784985.374306:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658381440:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 3909us (8937us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784985.374322:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 554 00000100:00100000:0.0:1787784985.374381:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 555 00000100:00100000:0.0:1787784985.374393:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658381696:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784985.374463:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658381696:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 74us (9095us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784985.374470:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 555 00000100:00100000:0.0:1787784985.374483:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 556 00000100:00100000:0.0:1787784985.374490:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658381312:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784985.374512:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658381312:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (9128us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784985.374519:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 556 00000100:00100000:0.0:1787784985.374530:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 557 00000100:00100000:0.0:1787784985.374548:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658381824:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784985.374571:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658381824:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 35us (4387us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784985.374577:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 557 00000100:00100000:0.0:1787784985.374659:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 579 00000100:00100000:0.0:1787784985.374683:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658381568:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784985.374751:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658381184:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 8887us (9427us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784985.374761:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 578 00000100:00080000:1.0:1787784985.374788:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787784985.374793:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784985.374796:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784985.374834:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658381568:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 151us (9430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784985.374841:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 579 00010000:00010000:2.0:1787784985.521448:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784985.857427:0:25062:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784985.857436:0:25062:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784985.858075:0:25062:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784985.858086:0:25062:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784986.545308:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787784987.122634:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784987.123577:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668482432:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784987.123646:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784987.127304:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482432 00000100:00100000:3.0:1787784987.127319:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482432 00000100:00100000:3.0:1787784987.127394:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:3.0:1787784987.127405:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668482432:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784987.127443:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784987.127535:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668482432:12345-0@lo:13:osp-pre-0-0.0 Request processed in 131us (3888us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784987.127542:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:3.0:1787784987.127601:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668482432:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787784987.129365:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787784987.573464:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784987.612030:0:25113:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784987.612039:0:25113:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784987.613864:0:25113:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784987.613878:0:25113:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784988.594227:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787784989.169306:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784989.169329:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668482560:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787784989.169394:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784989.169410:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940400a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668482816:0@lo:400:kworker.0 00000100:00100000:2.0:1787784989.169440:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784989.169442:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668482688:0@lo:400:kworker.0 00000100:00100000:2.0:1787784989.169458:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98940304ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668483328:0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.169468:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784989.169476:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784989.169488:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668483200:0@lo:400:kworker.0 00000100:00100000:2.0:1787784989.169492:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784989.169497:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668482944:0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.169507:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784989.169524:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482816 00000100:00100000:1.0:1787784989.169524:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787784989.169529:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f6aad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668483072:0@lo:400:kworker.0 00000100:00100000:2.0:1787784989.169530:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482816 00000100:00100000:1.0:1787784989.169551:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784989.169569:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483200 00000100:00100000:2.0:1787784989.169572:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483200 00000100:00100000:1.0:1787784989.169578:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482560 00000100:00100000:1.0:1787784989.169580:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482560 00000100:00100000:2.0:1787784989.169599:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 580 00000100:00100000:2.0:1787784989.169607:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668482816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.169625:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:1.0:1787784989.169632:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668482560:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787784989.169657:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784989.169666:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668482816:12345-0@lo:400:kworker.0 Request processed in 60us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784989.169673:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 580 00000100:00100000:1.0:1787784989.169689:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895392530c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668482560:12345-0@lo:13:osp-pre-1-0.0 Request processed in 59us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784989.169690:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 581 00000100:00100000:2.0:1787784989.169696:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668483200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.169696:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:2.0:1787784989.169722:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539253480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668483200:12345-0@lo:400:kworker.0 Request processed in 26us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784989.169726:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 581 00000100:00100000:2.0:1787784989.169743:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483328 00000100:00100000:2.0:1787784989.169746:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483328 00000100:00100000:2.0:1787784989.169767:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483072 00000100:00100000:2.0:1787784989.169769:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483072 00000100:00100000:2.0:1787784989.169790:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 558 00000100:00100000:2.0:1787784989.169796:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668483328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784989.169823:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668483328:12345-0@lo:400:kworker.0 Request processed in 28us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784989.169830:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 558 00000100:00100000:2.0:1787784989.169841:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 559 00000100:00100000:2.0:1787784989.169847:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668483072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.169869:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482688 00000100:00100000:1.0:1787784989.169873:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482688 00000100:00100000:1.0:1787784989.169896:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:1.0:1787784989.169903:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668482688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.170257:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668482688:12345-0@lo:400:kworker.0 Request processed in 354us (787us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784989.170265:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00100000:1.0:1787784989.170323:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894040092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668482688:0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.170336:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668483200:0@lo:400:kworker.0 00000100:00100000:1.0:1787784989.170350:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668482560:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784989.170754:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668483072:12345-0@lo:400:kworker.0 Request processed in 906us (1199us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784989.170762:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 559 00000100:00080000:2.0:1787784989.170787:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787784989.170791:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784989.170794:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787784989.170819:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784989.170898:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f6aad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668483072:0@lo:400:kworker.0 00000100:00100000:2.0:1787784989.171015:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940400a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668482816:0@lo:400:kworker.0 00000100:00100000:2.0:1787784989.171025:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98940304ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668483328:0@lo:400:kworker.0 00000100:00100000:3.0:1787784989.171683:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668482944 00000100:00100000:3.0:1787784989.171690:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668482944 00000100:00100000:3.0:1787784989.171874:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 582 00000100:00100000:3.0:1787784989.171885:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668482944:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787784989.171941:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787784989.171948:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668482944:12345-0@lo:400:kworker.0 Request processed in 66us (1129us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784989.171955:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 582 00000100:00100000:3.0:1787784989.171992:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668482944:0@lo:400:kworker.0 00000040:00080000:2.0:1787784989.397351:0:25165:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784989.397360:0:25165:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784989.398202:0:25165:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784989.398215:0:25165:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784989.617836:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784990.471437:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784990.471491:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784990.471534:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784990.471565:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784990.471601:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784990.471620:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784990.471641:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784990.471668:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382208 00000100:00100000:0.0:1787784990.471676:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382208 00000100:00100000:0.0:1787784990.471726:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382336 00000100:00100000:0.0:1787784990.471730:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382336 00000100:00100000:0.0:1787784990.471746:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382592 00000100:00100000:0.0:1787784990.471749:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382592 00000100:00100000:0.0:1787784990.471760:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382720 00000100:00100000:0.0:1787784990.471762:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382720 00000100:00100000:0.0:1787784990.471777:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 560 00000100:00100000:0.0:1787784990.471786:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658382208:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784990.471851:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658382208:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 66us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784990.471858:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 560 00000100:00100000:0.0:1787784990.471880:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 561 00000100:00100000:0.0:1787784990.471887:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658382336:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784990.471911:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658382336:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (346us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784990.471917:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 561 00000100:00100000:0.0:1787784990.471928:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 562 00000100:00100000:0.0:1787784990.471934:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658382592:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784990.471972:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658382592:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784990.471978:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 562 00000100:00100000:0.0:1787784990.471991:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:0.0:1787784990.471997:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658382720:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787784990.472022:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658382720:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784990.472029:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 563 00000100:00100000:2.0:1787784990.472726:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382080 00000100:00100000:2.0:1787784990.472733:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382080 00000100:00100000:2.0:1787784990.472779:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382464 00000100:00100000:2.0:1787784990.472783:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382464 00000100:00100000:2.0:1787784990.472806:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 583 00000100:00100000:2.0:1787784990.472814:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658382080:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784990.472857:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658382080:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 43us (1365us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784990.472864:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 583 00000100:00100000:2.0:1787784990.472879:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00100000:2.0:1787784990.472886:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658382464:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784990.472910:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063352c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658382464:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (1309us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784990.472917:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00080000:2.0:1787784990.472941:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787784990.472946:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787784990.472948:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784990.474928:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658381952 00000100:00100000:1.0:1787784990.474938:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658381952 00000100:00100000:1.0:1787784990.475008:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 653 00000100:00100000:1.0:1787784990.475018:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658381952:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784990.475117:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658381952:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 101us (3690us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784990.475125:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 653 00010000:00010000:1.0:1787784990.642262:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784991.220608:0:25216:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784991.220618:0:25216:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784991.221117:0:25216:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784991.221126:0:25216:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787784991.665316:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787784992.241418:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787784992.241441:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668483456:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787784992.241521:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784992.241664:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483456 00000100:00100000:1.0:1787784992.241672:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483456 00000100:00100000:1.0:1787784992.241734:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:1.0:1787784992.241744:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668483456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787784992.241778:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787784992.241832:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668483456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784992.241840:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00080000:2.0:1787784992.241904:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784992.242037:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668483456:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787784992.689262:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787784993.108869:0:25267:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787784993.108878:0:25267:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787784993.114929:0:25267:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787784993.114946:0:25267:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787784993.713746:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787784993.970552:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787784993.970561:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787784993.972507:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787784993.972526:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784994.292711:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539128b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668483840:0@lo:400:kworker.0 00000100:00100000:2.0:1787784994.292789:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784994.292841:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787784994.292848:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402adf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668483968:0@lo:400:kworker.0 00000100:00100000:2.0:1787784994.292875:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784994.292916:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483968 00000100:00100000:2.0:1787784994.292922:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483968 00000100:00100000:2.0:1787784994.292973:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 564 00000100:00100000:2.0:1787784994.292982:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668483968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784994.293030:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668483968:12345-0@lo:400:kworker.0 Request processed in 48us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784994.293038:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 564 00000100:00080000:2.0:1787784994.293058:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787784994.293095:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402adf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668483968:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293480:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668483712:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293515:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784994.293546:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668484224:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293569:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784994.293594:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1787784994.293600:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668483584:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293625:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784994.293646:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668484096:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293667:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784994.293697:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483840 00000100:00100000:1.0:1787784994.293700:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483840 00000100:00100000:1.0:1787784994.293733:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483712 00000100:00100000:1.0:1787784994.293735:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483712 00000100:00100000:1.0:1787784994.293767:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484096 00000100:00100000:1.0:1787784994.293770:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484096 00000100:00100000:1.0:1787784994.293784:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 585 00000100:00100000:1.0:1787784994.293791:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668483840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293842:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668483840:12345-0@lo:400:kworker.0 Request processed in 51us (1053us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784994.293848:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 585 00000100:00100000:1.0:1787784994.293861:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:1.0:1787784994.293866:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668483712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293896:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668483712:12345-0@lo:400:kworker.0 Request processed in 30us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784994.293902:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:1.0:1787784994.293913:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 587 00000100:00100000:1.0:1787784994.293919:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668484096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.293943:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668484096:12345-0@lo:400:kworker.0 Request processed in 24us (275us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784994.293948:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 587 00000100:00100000:1.0:1787784994.293969:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484224 00000100:00100000:1.0:1787784994.293973:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484224 00000100:00100000:1.0:1787784994.293994:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 565 00000100:00100000:1.0:1787784994.294001:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668484224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.294029:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668484224:12345-0@lo:400:kworker.0 Request processed in 28us (459us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784994.294036:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 565 00000100:00100000:1.0:1787784994.294053:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668483584 00000100:00100000:1.0:1787784994.294056:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668483584 00000100:00100000:1.0:1787784994.294077:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:1.0:1787784994.294084:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668483584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.294113:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668483584:12345-0@lo:400:kworker.0 Request processed in 30us (486us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784994.294129:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00080000:1.0:1787784994.294238:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787784994.294242:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784994.294261:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668483584:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.294272:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668484096:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.294280:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668483712:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.294288:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668484224:0@lo:400:kworker.0 00000100:00100000:1.0:1787784994.294325:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539128b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668483840:0@lo:400:kworker.0 00000100:00100000:2.0:1787784994.294764:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787784994.294775:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668484352:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787784994.294809:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784994.296881:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484352 00000100:00100000:2.0:1787784994.296890:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484352 00000100:00100000:2.0:1787784994.296944:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:2.0:1787784994.296953:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668484352:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784994.296992:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784994.297065:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402892c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668484352:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (2254us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784994.297072:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:2.0:1787784994.297137:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668484352:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1787784994.738348:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784995.066691:0:25319:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784995.066701:0:25319:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784995.067444:0:25319:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784995.067455:0:25319:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787784995.592346:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784995.592408:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382848 00000100:00100000:2.0:1787784995.592418:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382848 00000100:00100000:0.0:1787784995.592448:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787784995.592477:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 655 00000100:00100000:2.0:1787784995.592487:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658382848:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784995.592563:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b71450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658382848:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (228us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784995.592571:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 655 00000100:00100000:3.0:1787784995.592815:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658382976 00000100:00100000:3.0:1787784995.592820:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658382976 00000100:00100000:3.0:1787784995.592873:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00100000:3.0:1787784995.592881:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658382976:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784995.592929:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658382976:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 49us (480us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784995.592936:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00080000:1.0:1787784995.592997:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784995.594566:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784995.594599:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784995.594633:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784995.594654:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658383104 00000100:00100000:1.0:1787784995.594662:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658383104 00000100:00100000:3.0:1787784995.594718:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658383232 00000100:00100000:3.0:1787784995.594724:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658383232 00000100:00100000:1.0:1787784995.594731:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 566 00000100:00100000:1.0:1787784995.594744:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658383104:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784995.594768:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 567 00000100:00100000:3.0:1787784995.594779:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658383232:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787784995.594801:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658383104:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784995.594810:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 566 00000100:00100000:3.0:1787784995.594816:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658383232:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784995.594824:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 567 00000100:00100000:2.0:1787784995.594888:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658383360 00000100:00100000:2.0:1787784995.594892:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658383360 00000100:00100000:2.0:1787784995.594932:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00100000:2.0:1787784995.594940:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658383360:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787784995.594972:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658383360:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784995.594978:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00080000:0.0:1787784995.595086:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787784995.595090:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787784995.595092:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787784995.599647:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787784995.599950:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787784995.600836:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658383488 00000100:00100000:1.0:1787784995.600847:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658383488 00000100:00080000:0.0:1787784995.601038:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787784995.601187:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658383616 00000100:00100000:1.0:1787784995.601192:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658383616 00000100:00080000:0.0:1787784995.601247:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784995.601343:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 568 00000100:00100000:3.0:1787784995.601355:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658383488:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784995.601481:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658383488:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 130us (1845us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784995.601489:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 568 00000100:00100000:3.0:1787784995.601508:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 569 00000100:00100000:3.0:1787784995.601515:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401837c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658383616:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787784995.601552:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401837c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658383616:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (1603us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784995.601558:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 569 00010000:00010000:2.0:1787784995.761417:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787784996.785360:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784996.989879:0:25370:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784996.989889:0:25370:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784996.990848:0:25370:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784996.990860:0:25370:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787784997.361455:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668484480:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787784997.361527:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787784997.361687:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484480 00000100:00100000:3.0:1787784997.361695:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484480 00000100:00100000:3.0:1787784997.361760:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1787784997.361769:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668484480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787784997.361802:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787784997.361856:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895402883c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668484480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (329us total) trans 0 rc 0/0 00000100:00100000:3.0:1787784997.361864:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1787784997.361912:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668484480:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787784997.361976:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787784997.810859:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787784998.789968:0:25421:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787784998.789977:0:25421:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787784998.798798:0:25421:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787784998.798816:0:25421:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787784998.836411:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787784999.409506:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787784999.409529:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668484608:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787784999.409607:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787784999.409896:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484608 00000100:00100000:2.0:1787784999.409905:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484608 00000100:00100000:2.0:1787784999.409963:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:2.0:1787784999.409972:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668484608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787784999.410017:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787784999.410063:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668484608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (456us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784999.410070:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00080000:3.0:1787784999.411660:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787784999.411733:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540288b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668484608:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787784999.414308:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894018361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668484992:0@lo:400:kworker.0 00000100:00100000:0.0:1787784999.414394:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784999.414447:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787784999.414454:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668484864:0@lo:400:kworker.0 00000100:00100000:0.0:1787784999.414478:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787784999.414509:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484992 00000100:00100000:0.0:1787784999.414515:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484992 00000100:00100000:0.0:1787784999.414834:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484864 00000100:00100000:0.0:1787784999.414838:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484864 00000100:00100000:0.0:1787784999.414861:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 590 00000100:00100000:0.0:1787784999.414869:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668484992:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784999.414927:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668484992:12345-0@lo:400:kworker.0 Request processed in 58us (544us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784999.414934:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 590 00000100:00100000:0.0:1787784999.414956:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 591 00000100:00100000:0.0:1787784999.414961:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668484864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784999.414987:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668484864:12345-0@lo:400:kworker.0 Request processed in 26us (509us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784999.415059:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 591 00000100:00080000:0.0:1787784999.415136:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787784999.415189:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401834780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668484864:0@lo:400:kworker.0 00000100:00100000:0.0:1787784999.415199:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894018361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668484992:0@lo:400:kworker.0 00000100:00100000:1.0:1787784999.415345:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668485120:0@lo:400:kworker.0 00000100:00100000:1.0:1787784999.415381:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784999.415423:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1787784999.415429:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401835e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668484736:0@lo:400:kworker.0 00000100:00100000:1.0:1787784999.415462:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784999.415490:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668485248:0@lo:400:kworker.0 00000100:00100000:1.0:1787784999.415510:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787784999.415545:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668485248 00000100:00100000:1.0:1787784999.415549:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668485248 00000100:00100000:1.0:1787784999.415582:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 592 00000100:00100000:1.0:1787784999.415588:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668485248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787784999.415624:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668485248:12345-0@lo:400:kworker.0 Request processed in 36us (113us total) trans 0 rc 0/0 00000100:00100000:1.0:1787784999.415631:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 592 00000100:00080000:1.0:1787784999.415644:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787784999.415660:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989402a92940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668485248:0@lo:400:kworker.0 00000100:00100000:2.0:1787784999.415919:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668485120 00000100:00100000:2.0:1787784999.415924:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668485120 00000100:00100000:2.0:1787784999.415966:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 570 00000100:00100000:2.0:1787784999.415973:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668485120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787784999.416025:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668485120:12345-0@lo:400:kworker.0 Request processed in 53us (643us total) trans 0 rc 0/0 00000100:00100000:2.0:1787784999.416031:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 570 00000100:00100000:2.0:1787784999.416061:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668485120:0@lo:400:kworker.0 00000100:00100000:0.0:1787784999.416369:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668484736 00000100:00100000:0.0:1787784999.416374:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668484736 00000100:00100000:0.0:1787784999.416420:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:0.0:1787784999.416427:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668484736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787784999.416475:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b72850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668484736:12345-0@lo:400:kworker.0 Request processed in 48us (1011us total) trans 0 rc 0/0 00000100:00100000:0.0:1787784999.416481:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00100000:3.0:1787784999.416714:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401835e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668484736:0@lo:400:kworker.0 00000100:00080000:1.0:1787784999.417314:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787784999.857441:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785000.720435:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785000.720491:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785000.720494:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658383872 00000100:00100000:2.0:1787785000.720511:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658383872 00000100:00100000:3.0:1787785000.720539:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785000.720587:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785000.720597:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 593 00000100:00100000:2.0:1787785000.720610:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658383872:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785000.720620:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785000.720644:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785000.720676:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785000.720707:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658383744 00000100:00100000:3.0:1787785000.720712:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658383744 00000100:00100000:2.0:1787785000.720717:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658383872:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 108us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785000.720726:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 593 00000100:00100000:2.0:1787785000.720748:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384256 00000100:00100000:2.0:1787785000.720751:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384256 00000100:00100000:3.0:1787785000.720767:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 657 00000100:00100000:3.0:1787785000.720778:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658383744:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785000.720785:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 594 00000100:00100000:2.0:1787785000.720796:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658384256:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785000.720818:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b73850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658383744:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (228us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785000.720825:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 657 00000100:00100000:2.0:1787785000.720833:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658384256:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785000.720841:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 594 00000100:00100000:2.0:1787785000.720874:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384128 00000100:00100000:2.0:1787785000.720878:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384128 00000100:00100000:2.0:1787785000.720903:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384384 00000100:00100000:2.0:1787785000.720906:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384384 00000100:00100000:2.0:1787785000.720922:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384000 00000100:00100000:2.0:1787785000.720925:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384000 00000100:00100000:2.0:1787785000.720938:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384512 00000100:00100000:2.0:1787785000.720940:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384512 00000100:00100000:2.0:1787785000.720956:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 571 00000100:00100000:2.0:1787785000.720963:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658384128:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785000.720995:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658384128:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (504us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785000.721001:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 571 00000100:00100000:2.0:1787785000.721013:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 572 00000100:00100000:2.0:1787785000.721019:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658384384:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785000.721043:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658384384:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785000.721049:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 572 00000100:00100000:2.0:1787785000.721058:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:2.0:1787785000.721064:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658384000:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785000.721098:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658384000:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785000.721105:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 573 00000100:00100000:2.0:1787785000.721115:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 574 00000100:00100000:2.0:1787785000.721121:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658384512:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785000.721141:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658384512:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785000.721171:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 574 00000100:00080000:1.0:1787785000.721443:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787785000.721449:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785000.721452:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:3.0:1787785000.837274:0:25472:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785000.837289:0:25472:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785000.837871:0:25472:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785000.837884:0:25472:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785000.881709:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787785001.905783:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787785002.482104:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787785002.482139:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668485376:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787785002.482270:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785002.482433:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668485376 00000100:00100000:3.0:1787785002.482444:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668485376 00000100:00100000:3.0:1787785002.482517:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:3.0:1787785002.482529:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668485376:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787785002.482576:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787785002.482632:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401835e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668485376:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (363us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785002.482640:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00080000:0.0:1787785002.482824:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785002.482871:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668485376:0@lo:13:osp-pre-0-0.0 00000040:00080000:1.0:1787785002.927694:0:25524:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785002.927707:0:25524:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785002.932582:0:25524:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785002.932621:0:25524:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787785002.935524:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785003.189739:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785003.189751:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785003.195670:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785003.195700:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785003.953656:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787785004.530969:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668485504:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.531081:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785004.531174:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668486016:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.531211:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785004.531224:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668485504 00000100:00100000:3.0:1787785004.531237:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668485504 00000100:00100000:0.0:1787785004.531246:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787785004.531255:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668485888:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.531292:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785004.531300:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:3.0:1787785004.531315:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668485504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.531331:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668485632:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787785004.531373:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785004.531394:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989507b70850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668485504:12345-0@lo:400:kworker.0 Request processed in 82us (313us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785004.531403:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668486144:0@lo:400:kworker.0 00000100:00100000:3.0:1787785004.531406:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00000100:00100000:0.0:1787785004.531440:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785004.531461:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1787785004.531467:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668485760:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.531491:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785004.531516:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668486272:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.531537:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785004.531585:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668485888 00000100:00100000:0.0:1787785004.531589:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668485888 00000100:00100000:2.0:1787785004.531718:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486016 00000100:00100000:2.0:1787785004.531724:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486016 00000100:00100000:2.0:1787785004.531761:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486272 00000100:00100000:2.0:1787785004.531764:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486272 00000100:00100000:2.0:1787785004.531789:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:2.0:1787785004.531796:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668486016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785004.531838:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894018361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668486016:12345-0@lo:400:kworker.0 Request processed in 42us (627us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785004.531846:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:2.0:1787785004.531859:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 576 00000100:00100000:2.0:1787785004.531866:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668486272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785004.531901:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668486272:12345-0@lo:400:kworker.0 Request processed in 35us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785004.531908:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 576 00000100:00100000:0.0:1787785004.531950:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486144 00000100:00100000:0.0:1787785004.531954:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486144 00000100:00100000:3.0:1787785004.536584:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668485760 00000100:00100000:3.0:1787785004.536604:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668485760 00000100:00100000:3.0:1787785004.536778:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 595 00000100:00100000:3.0:1787785004.536796:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668485888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785004.536923:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401834780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668485888:12345-0@lo:400:kworker.0 Request processed in 136us (5631us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785004.536931:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 595 00000100:00100000:3.0:1787785004.536963:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 596 00000100:00100000:3.0:1787785004.536971:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668486144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785004.537006:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668486144:12345-0@lo:400:kworker.0 Request processed in 36us (5566us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785004.537012:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 596 00000100:00100000:3.0:1787785004.537030:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 597 00000100:00100000:3.0:1787785004.537037:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668485760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785004.537125:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668485760:12345-0@lo:400:kworker.0 Request processed in 88us (5633us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785004.537132:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 597 00000100:00100000:2.0:1787785004.546026:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668485760:0@lo:400:kworker.0 00000100:00100000:2.0:1787785004.546068:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668486272:0@lo:400:kworker.0 00000100:00100000:2.0:1787785004.546085:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668486144:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.551339:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989404dbe1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668485888:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.551362:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f09ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668485504:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.551378:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b536940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668486016:0@lo:400:kworker.0 00000100:00100000:0.0:1787785004.551432:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668485632 00000100:00100000:0.0:1787785004.551442:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668485632 00000100:00100000:0.0:1787785004.551500:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1787785004.551511:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668485632:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785004.551565:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785004.551694:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668485632:12345-0@lo:13:osp-pre-1-0.0 Request processed in 184us (20320us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785004.551702:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00080000:0.0:1787785004.551745:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787785004.551751:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787785004.551755:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785004.552379:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401836940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668485632:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1787785004.986360:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785005.171032:0:25575:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785005.171058:0:25575:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785005.171637:0:25575:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785005.171647:0:25575:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785005.832630:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785005.832687:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785005.832719:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785005.832750:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785005.832768:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785005.832797:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785005.832823:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785005.832855:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385024 00000100:00100000:0.0:1787785005.832868:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385024 00000100:00100000:0.0:1787785005.832932:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385280 00000100:00100000:0.0:1787785005.832935:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385280 00000100:00100000:0.0:1787785005.832951:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384896 00000100:00100000:0.0:1787785005.832954:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384896 00000100:00100000:0.0:1787785005.832967:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385408 00000100:00100000:0.0:1787785005.832969:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385408 00000100:00100000:0.0:1787785005.832984:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 577 00000100:00100000:0.0:1787785005.833360:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658385024:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785005.833459:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658385024:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 107us (771us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785005.833542:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384768 00000100:00100000:2.0:1787785005.833548:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384768 00000100:00100000:2.0:1787785005.833589:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385152 00000100:00100000:2.0:1787785005.833592:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385152 00000100:00100000:2.0:1787785005.833615:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 598 00000100:00100000:2.0:1787785005.833623:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658384768:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785005.833660:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658384768:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 38us (1045us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785005.833667:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 598 00000100:00100000:2.0:1787785005.833688:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:2.0:1787785005.833695:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658385152:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785005.833722:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989404dbe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658385152:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (924us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785005.833728:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:3.0:1787785005.834391:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658384640 00000100:00100000:3.0:1787785005.834400:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658384640 00000100:00100000:3.0:1787785005.834448:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 659 00000100:00100000:3.0:1787785005.834456:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658384640:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785005.834500:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658384640:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 46us (1749us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785005.834508:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 659 00000100:00100000:1.0:1787785005.836254:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 578 00000100:00100000:1.0:1787785005.836289:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658385280:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785005.836382:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658385280:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 111us (3660us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785005.836393:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 578 00000100:00100000:1.0:1787785005.836518:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 579 00000100:00100000:1.0:1787785005.836528:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658384896:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785005.836557:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658384896:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (3788us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785005.836564:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 579 00000100:00100000:1.0:1787785005.836579:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 580 00000100:00100000:1.0:1787785005.836588:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658385408:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785005.836614:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658385408:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (3790us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785005.836620:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 580 00000100:00100000:0.0:1787785005.838328:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 577 00000100:00080000:0.0:1787785005.838454:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787785005.838462:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787785005.838466:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787785006.001417:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787785007.025429:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785007.217966:0:25626:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785007.217979:0:25626:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785007.218526:0:25626:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785007.218538:0:25626:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785007.604219:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787785007.604258:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668486400:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787785007.604376:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785007.604826:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486400 00000100:00100000:3.0:1787785007.604848:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486400 00000100:00100000:3.0:1787785007.604940:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:3.0:1787785007.604952:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035961c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668486400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787785007.605001:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787785007.605330:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035961c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668486400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 378us (952us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785007.605339:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00080000:0.0:1787785007.605475:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787785007.607193:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b535e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668486400:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787785008.052044:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787785009.073308:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785009.366900:0:25678:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785009.366914:0:25678:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785009.370697:0:25678:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785009.370725:0:25678:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787785009.651679:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668486528:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785009.651781:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785009.651884:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486528 00000100:00100000:2.0:1787785009.651894:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486528 00000100:00100000:2.0:1787785009.651967:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1787785009.651977:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668486528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785009.652036:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785009.652089:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668486528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (308us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785009.652096:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00080000:2.0:1787785009.652122:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787785009.652382:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668486784:0@lo:400:kworker.0 00000100:00100000:1.0:1787785009.652433:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785009.652466:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1787785009.652474:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668486656:0@lo:400:kworker.0 00000100:00100000:1.0:1787785009.652809:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785009.652840:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668487168:0@lo:400:kworker.0 00000100:00100000:1.0:1787785009.652867:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785009.652905:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486784 00000100:00100000:1.0:1787785009.652910:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486784 00000100:00100000:1.0:1787785009.652949:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487168 00000100:00100000:1.0:1787785009.652953:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487168 00000100:00100000:1.0:1787785009.652976:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 600 00000100:00100000:1.0:1787785009.652984:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668486784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785009.653041:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668486784:12345-0@lo:400:kworker.0 Request processed in 58us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785009.653048:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 600 00000100:00100000:1.0:1787785009.653061:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 601 00000100:00100000:1.0:1787785009.653068:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668487168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785009.653094:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668487168:12345-0@lo:400:kworker.0 Request processed in 28us (227us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785009.653099:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 601 00000100:00100000:1.0:1787785009.653134:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668487168:0@lo:400:kworker.0 00000100:00100000:3.0:1787785009.655264:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486656 00000100:00100000:3.0:1787785009.655286:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486656 00000100:00100000:3.0:1787785009.655370:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:3.0:1787785009.655382:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668486656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785009.655503:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668486656:12345-0@lo:400:kworker.0 Request processed in 123us (2694us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785009.655510:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00100000:2.0:1787785009.655758:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668486528:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785009.655796:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668487040:0@lo:400:kworker.0 00000100:00100000:2.0:1787785009.655838:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785009.655884:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787785009.655892:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401fa6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668486912:0@lo:400:kworker.0 00000100:00100000:2.0:1787785009.655919:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785009.655958:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668486912 00000100:00100000:2.0:1787785009.655961:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668486912 00000100:00100000:2.0:1787785009.656003:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:2.0:1787785009.656023:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668486912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785009.656066:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668486912:12345-0@lo:400:kworker.0 Request processed in 56us (146us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785009.656074:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00080000:2.0:1787785009.656122:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787785009.656182:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668486912:0@lo:400:kworker.0 00000100:00100000:1.0:1787785009.656336:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668486784:0@lo:400:kworker.0 00000100:00100000:1.0:1787785009.656370:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668486656:0@lo:400:kworker.0 00000100:00100000:3.0:1787785009.657056:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487040 00000100:00100000:3.0:1787785009.657065:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487040 00000100:00100000:3.0:1787785009.657127:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 581 00000100:00100000:3.0:1787785009.657136:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668487040:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787785009.657319:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785009.657678:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668487040:12345-0@lo:400:kworker.0 Request processed in 541us (1837us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785009.657689:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 581 00000100:00100000:2.0:1787785009.657873:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401fa4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668487040:0@lo:400:kworker.0 00010000:00010000:1.0:1787785010.097419:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785010.955325:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785010.955382:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785010.955417:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785010.955425:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385536 00000100:00100000:3.0:1787785010.955439:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785010.955439:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385536 00000100:00100000:3.0:1787785010.955457:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785010.955474:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785010.955501:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785010.955502:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 661 00000100:00100000:0.0:1787785010.955513:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658385536:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785010.955530:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385664 00000100:00100000:3.0:1787785010.955533:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385664 00000100:00100000:3.0:1787785010.955574:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386048 00000100:00100000:3.0:1787785010.955577:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386048 00000100:00100000:3.0:1787785010.955599:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 603 00000100:00100000:0.0:1787785010.955601:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658385536:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 90us (219us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785010.955606:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658385664:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785010.955609:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 661 00000100:00100000:3.0:1787785010.955638:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658385664:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (323us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785010.955644:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 603 00000100:00100000:3.0:1787785010.955655:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 604 00000100:00100000:0.0:1787785010.955655:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385792 00000100:00100000:0.0:1787785010.955659:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385792 00000100:00100000:3.0:1787785010.955661:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658386048:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785010.955684:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386304 00000100:00100000:3.0:1787785010.955685:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658386048:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (246us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785010.955687:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386304 00000100:00100000:3.0:1787785010.955691:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 604 00000100:00100000:0.0:1787785010.955705:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658385920 00000100:00100000:0.0:1787785010.955708:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658385920 00000100:00100000:3.0:1787785010.955711:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386176 00000100:00100000:3.0:1787785010.955714:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386176 00000100:00100000:0.0:1787785010.955723:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 582 00000100:00100000:0.0:1787785010.955729:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658385792:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785010.955731:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 583 00000100:00100000:3.0:1787785010.955737:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658386304:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785010.955756:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658385792:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785010.955757:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658386304:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785010.955763:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 582 00000100:00100000:3.0:1787785010.957780:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 583 00000100:00100000:3.0:1787785010.957823:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00100000:3.0:1787785010.957839:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658385920:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785010.957918:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658385920:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 86us (2442us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785010.957926:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 584 00000100:00100000:3.0:1787785010.957940:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 585 00000100:00100000:3.0:1787785010.957948:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658386176:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785010.957976:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401836940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658386176:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (2475us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785010.957982:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 585 00000100:00080000:3.0:1787785010.958021:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787785010.958028:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787785010.958032:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787785011.125529:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785011.317809:0:25729:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785011.317823:0:25729:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785011.318438:0:25729:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785011.318459:0:25729:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787785012.145505:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785012.721645:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787785012.721681:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668487296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787785012.721786:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785012.721912:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487296 00000100:00100000:3.0:1787785012.721923:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487296 00000100:00100000:3.0:1787785012.722011:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:3.0:1787785012.722023:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668487296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787785012.722074:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787785012.722134:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540288b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668487296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 114us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785012.722142:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00080000:2.0:1787785012.722312:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785012.722374:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668487296:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1787785013.178319:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785013.386891:0:25781:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785013.386901:0:25781:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785013.388384:0:25781:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785013.388482:0:25781:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:0.0:1787785013.489457:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785013.489471:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785013.489976:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785013.490007:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785014.193593:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787785014.769370:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787785014.769395:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668487424:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787785014.769466:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785014.769561:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487424 00000100:00100000:2.0:1787785014.769569:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487424 00000100:00100000:2.0:1787785014.769647:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 259 00000100:00100000:2.0:1787785014.769657:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668487424:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785014.769693:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785014.769744:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668487424:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785014.769750:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 259 00000100:00100000:2.0:1787785014.769795:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668487424:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1787785014.769851:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787785014.770479:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668487936:0@lo:400:kworker.0 00000100:00100000:0.0:1787785014.770516:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785014.770552:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1787785014.770567:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668487552:0@lo:400:kworker.0 00000100:00100000:0.0:1787785014.770597:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785014.770617:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668488064:0@lo:400:kworker.0 00000100:00100000:0.0:1787785014.770641:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785014.770752:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488064 00000100:00100000:2.0:1787785014.770757:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488064 00000100:00100000:2.0:1787785014.770805:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:2.0:1787785014.770813:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668488064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785014.770855:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668488064:12345-0@lo:400:kworker.0 Request processed in 43us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785014.770863:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00100000:2.0:1787785014.770896:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668488064:0@lo:400:kworker.0 00000100:00080000:0.0:1787785014.770955:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785014.771008:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487936 00000100:00100000:3.0:1787785014.771013:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487936 00000100:00100000:3.0:1787785014.771065:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1787785014.771074:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668487936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785014.771113:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668487936:12345-0@lo:400:kworker.0 Request processed in 41us (596us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785014.771119:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1787785014.771140:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487552 00000100:00100000:3.0:1787785014.771143:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487552 00000100:00100000:3.0:1787785014.771195:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:3.0:1787785014.771200:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668487552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785014.771232:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668487552:12345-0@lo:400:kworker.0 Request processed in 31us (634us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785014.771238:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00080000:0.0:1787785014.771322:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785014.771381:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5be940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668487552:0@lo:400:kworker.0 00000100:00100000:2.0:1787785014.771392:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668487936:0@lo:400:kworker.0 00000100:00100000:1.0:1787785014.771569:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668487680:0@lo:400:kworker.0 00000100:00100000:1.0:1787785014.771602:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785014.771633:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787785014.771639:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668487808:0@lo:400:kworker.0 00000100:00100000:1.0:1787785014.771660:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785014.771735:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487680 00000100:00100000:0.0:1787785014.771739:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487680 00000100:00100000:0.0:1787785014.771774:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668487808 00000100:00100000:0.0:1787785014.771778:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668487808 00000100:00100000:0.0:1787785014.771803:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:0.0:1787785014.771809:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668487680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785014.771848:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668487680:12345-0@lo:400:kworker.0 Request processed in 39us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785014.771855:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:0.0:1787785014.771872:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 607 00000100:00100000:0.0:1787785014.771878:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668487808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785014.771904:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668487808:12345-0@lo:400:kworker.0 Request processed in 27us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785014.771910:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 607 00000100:00080000:0.0:1787785014.771924:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787785014.772260:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668487808:0@lo:400:kworker.0 00000100:00100000:1.0:1787785014.772272:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953f7b1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668487680:0@lo:400:kworker.0 00010000:00010000:2.0:1787785015.218470:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785015.367497:0:25832:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785015.367505:0:25832:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785015.370745:0:25832:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785015.370770:0:25832:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785016.073310:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785016.073356:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785016.073391:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785016.073422:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785016.073448:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785016.073472:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386688 00000100:00100000:0.0:1787785016.073475:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785016.073483:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386688 00000100:00100000:0.0:1787785016.073507:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785016.073535:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387200 00000100:00100000:0.0:1787785016.073539:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387200 00000100:00100000:2.0:1787785016.073546:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386816 00000100:00100000:2.0:1787785016.073552:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386816 00000100:00100000:3.0:1787785016.073606:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386944 00000100:00100000:3.0:1787785016.073611:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386944 00000100:00100000:1.0:1787785016.073642:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386432 00000100:00100000:1.0:1787785016.073660:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386432 00000100:00100000:1.0:1787785016.073695:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:1.0:1787785016.073705:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658386432:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785016.073774:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658386432:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 69us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785016.073782:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:1.0:1787785016.073814:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387072 00000100:00100000:1.0:1787785016.073818:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387072 00000100:00100000:0.0:1787785016.073923:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 587 00000100:00100000:0.0:1787785016.073931:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658386688:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785016.073964:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403595e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658386688:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785016.073981:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 587 00000100:00100000:0.0:1787785016.073995:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00100000:0.0:1787785016.074001:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658387200:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785016.074026:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403596940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658387200:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (603us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785016.074032:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 588 00000100:00100000:0.0:1787785016.074055:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658386560 00000100:00100000:0.0:1787785016.074059:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658386560 00000100:00100000:2.0:1787785016.075678:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00100000:2.0:1787785016.075692:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658386816:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785016.075853:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658386816:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 163us (2377us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785016.075865:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 589 00000100:00080000:2.0:1787785016.076955:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787785016.076963:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785016.079342:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 608 00000100:00100000:3.0:1787785016.079359:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658386944:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785016.079516:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403594780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658386944:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 161us (6122us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785016.079523:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 608 00000100:00080000:2.0:1787785016.080059:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787785016.080847:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 590 00000100:00100000:1.0:1787785016.080861:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658387072:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:2.0:1787785016.080918:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787785016.080932:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658387072:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 74us (7423us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785016.080940:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 590 00000100:00100000:0.0:1787785016.082901:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 609 00000100:00100000:0.0:1787785016.082916:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658386560:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:2.0:1787785016.083090:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785016.083781:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402185e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658386560:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 865us (10328us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785016.083791:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 609 00010000:00010000:1.0:1787785016.244278:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785017.150352:0:25884:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785017.150362:0:25884:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785017.150906:0:25884:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785017.150915:0:25884:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785017.268985:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785017.841901:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787785017.841925:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668488192:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787785017.842006:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785017.842337:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488192 00000100:00100000:3.0:1787785017.842345:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488192 00000100:00100000:3.0:1787785017.842414:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:3.0:1787785017.842423:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668488192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787785017.842457:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787785017.842517:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668488192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785017.842524:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00080000:2.0:1787785017.842585:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785017.842663:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668488192:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787785018.293888:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785019.015564:0:25935:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785019.015575:0:25935:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785019.016114:0:25935:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785019.016123:0:25935:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787785019.315854:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785019.891900:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668488448:0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.891986:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785019.891991:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668488320:0@lo:400:kworker.0 00000100:00100000:1.0:1787785019.892041:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785019.892055:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98951b5bed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668488960:0@lo:400:kworker.0 00000100:00100000:1.0:1787785019.892073:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668488832:0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.892089:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785019.892108:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785019.892118:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787785019.892126:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668488576:0@lo:400:kworker.0 00000100:00100000:1.0:1787785019.892130:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787785019.892139:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668488704:0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.892173:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785019.892209:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785019.892210:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488448 00000100:00100000:2.0:1787785019.892218:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488448 00000100:00100000:1.0:1787785019.892261:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488960 00000100:00100000:1.0:1787785019.892266:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488960 00000100:00100000:2.0:1787785019.892278:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488832 00000100:00100000:2.0:1787785019.892283:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488832 00000100:00100000:1.0:1787785019.892312:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488704 00000100:00100000:2.0:1787785019.892316:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488576 00000100:00100000:1.0:1787785019.892317:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488704 00000100:00100000:2.0:1787785019.892321:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488576 00000100:00100000:2.0:1787785019.892348:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:1.0:1787785019.892352:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 591 00000100:00100000:2.0:1787785019.892358:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668488448:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785019.892364:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668488960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785019.892421:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668488960:12345-0@lo:400:kworker.0 Request processed in 61us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785019.892423:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5be940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668488448:12345-0@lo:400:kworker.0 Request processed in 67us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785019.892433:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 591 00000100:00100000:2.0:1787785019.892436:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:1.0:1787785019.892456:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 592 00000100:00100000:2.0:1787785019.892460:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 611 00000100:00100000:1.0:1787785019.892467:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668488704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.892471:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668488832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785019.892513:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668488704:12345-0@lo:400:kworker.0 Request processed in 49us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785019.892519:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668488832:12345-0@lo:400:kworker.0 Request processed in 51us (410us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785019.892524:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 592 00000100:00100000:2.0:1787785019.892532:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 611 00000100:00100000:2.0:1787785019.892561:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 612 00000100:00100000:1.0:1787785019.892569:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668488704:0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.892570:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668488576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785019.892590:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668488832:0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.892608:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f09cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668488576:12345-0@lo:400:kworker.0 Request processed in 41us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785019.892615:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 612 00000100:00080000:1.0:1787785019.892642:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787785019.892649:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787785019.892651:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787785019.892668:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668488576:0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.892681:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540289e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668488448:0@lo:400:kworker.0 00000100:00100000:2.0:1787785019.892689:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98951b5bed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668488960:0@lo:400:kworker.0 00000100:00100000:3.0:1787785019.893026:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668488320 00000100:00100000:3.0:1787785019.893031:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668488320 00000100:00100000:3.0:1787785019.893065:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:3.0:1787785019.893072:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668488320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785019.893120:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668488320:12345-0@lo:400:kworker.0 Request processed in 49us (1080us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785019.893126:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:1.0:1787785019.894355:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787785019.894374:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668489088:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787785019.894424:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785019.894693:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489088 00000100:00100000:0.0:1787785019.894700:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489088 00000100:00100000:0.0:1787785019.894742:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:0.0:1787785019.894751:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668489088:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785019.894782:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785019.894835:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401fa5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668489088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785019.894842:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:3.0:1787785019.894930:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668489088:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785019.899681:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668488320:0@lo:400:kworker.0 00010000:00010000:3.0:1787785020.337867:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785020.807717:0:25986:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785020.807725:0:25986:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785020.808429:0:25986:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785020.808440:0:25986:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785021.192422:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785021.192466:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785021.192505:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785021.192539:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785021.192559:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785021.192583:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785021.192607:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785021.192636:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387840 00000100:00100000:3.0:1787785021.192645:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387840 00000100:00100000:3.0:1787785021.192701:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387456 00000100:00100000:3.0:1787785021.192705:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387456 00000100:00100000:3.0:1787785021.192729:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 613 00000100:00100000:3.0:1787785021.192739:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658387840:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785021.192752:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 614 00000100:00100000:3.0:1787785021.192808:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cede00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658387840:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 71us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785021.192817:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 613 00000100:00100000:1.0:1787785021.192829:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658387456:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785021.192841:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387584 00000100:00100000:3.0:1787785021.192858:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387584 00000100:00100000:1.0:1787785021.192864:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658387456:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 102us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785021.192872:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 614 00000100:00100000:3.0:1787785021.192889:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388096 00000100:00100000:3.0:1787785021.192893:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388096 00000100:00100000:1.0:1787785021.192898:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387712 00000100:00100000:1.0:1787785021.192903:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387712 00000100:00100000:3.0:1787785021.192918:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387968 00000100:00100000:3.0:1787785021.192922:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387968 00000100:00100000:2.0:1787785021.193038:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658387328 00000100:00100000:2.0:1787785021.193042:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658387328 00000100:00100000:1.0:1787785021.193053:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 593 00000100:00100000:1.0:1787785021.193065:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658387584:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785021.193076:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 665 00000100:00100000:2.0:1787785021.193086:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658387328:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785021.193097:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658387584:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (629us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785021.193139:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658387328:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 54us (727us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785021.193195:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 594 00000100:00100000:3.0:1787785021.193204:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658388096:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785021.193236:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658388096:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (697us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785021.193243:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 594 00000100:00100000:3.0:1787785021.193256:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 595 00000100:00100000:3.0:1787785021.193267:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658387712:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785021.193290:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658387712:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (707us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785021.193296:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 595 00000100:00100000:2.0:1787785021.193345:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 665 00000100:00100000:2.0:1787785021.193379:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 596 00000100:00100000:2.0:1787785021.193386:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658387968:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785021.193411:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401cef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658387968:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (804us total) trans 0 rc 0/0 00000100:00080000:3.0:1787785021.193452:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787785021.193457:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787785021.193460:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787785021.193503:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 596 00000100:00100000:1.0:1787785021.194103:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 593 00010000:00010000:0.0:1787785021.363675:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787785022.392283:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785022.636660:0:26037:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785022.636669:0:26037:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785022.642593:0:26037:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785022.642614:0:26037:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787785022.963019:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668489216:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787785022.963091:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785022.963277:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489216 00000100:00100000:0.0:1787785022.963286:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489216 00000100:00100000:0.0:1787785022.963374:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:0.0:1787785022.963383:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668489216:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787785022.963418:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785022.963465:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668489216:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785022.963472:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:0.0:1787785022.963519:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668489216:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787785022.963721:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787785023.409461:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785023.729856:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785023.729865:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785023.730868:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785023.730883:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785024.435325:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785024.533884:0:26089:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785024.533892:0:26089:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785024.542312:0:26089:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785024.542331:0:26089:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787785025.012258:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787785025.012281:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668489344:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787785025.012351:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785025.012647:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668489728:0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.012680:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785025.012701:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787785025.012706:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668489856:0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.012731:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785025.012764:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489728 00000100:00100000:3.0:1787785025.012770:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489728 00000100:00100000:3.0:1787785025.012822:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:3.0:1787785025.012833:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668489728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.012885:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540738f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668489728:12345-0@lo:400:kworker.0 Request processed in 54us (203us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785025.012891:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:3.0:1787785025.012925:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489856 00000100:00100000:3.0:1787785025.012929:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489856 00000100:00100000:3.0:1787785025.012955:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 597 00000100:00100000:3.0:1787785025.012962:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668489856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.012993:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668489856:12345-0@lo:400:kworker.0 Request processed in 32us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785025.012999:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 597 00000100:00080000:3.0:1787785025.013014:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787785025.013019:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785025.013049:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401ceed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668489856:0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.013061:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668489728:0@lo:400:kworker.0 00000100:00100000:2.0:1787785025.013134:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668489600:0@lo:400:kworker.0 00000100:00100000:2.0:1787785025.013197:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785025.013218:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787785025.013224:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668489472:0@lo:400:kworker.0 00000100:00100000:2.0:1787785025.013251:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785025.013275:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668489984:0@lo:400:kworker.0 00000100:00100000:2.0:1787785025.013297:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785025.013325:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489472 00000100:00100000:2.0:1787785025.013329:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489472 00000100:00100000:2.0:1787785025.013359:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:2.0:1787785025.013365:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668489472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785025.013399:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668489472:12345-0@lo:400:kworker.0 Request processed in 34us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785025.013406:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00100000:2.0:1787785025.013446:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668489472:0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.013550:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489600 00000100:00100000:3.0:1787785025.013554:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489600 00000100:00100000:3.0:1787785025.013585:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489984 00000100:00100000:3.0:1787785025.013589:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489984 00000100:00100000:3.0:1787785025.013623:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:3.0:1787785025.013629:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668489600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.013666:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5bed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668489600:12345-0@lo:400:kworker.0 Request processed in 37us (468us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785025.013672:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00100000:3.0:1787785025.013685:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:3.0:1787785025.013691:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668489984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.013718:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402adf0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668489984:12345-0@lo:400:kworker.0 Request processed in 28us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785025.013724:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00080000:1.0:1787785025.013783:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785025.013844:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668489344 00000100:00100000:0.0:1787785025.013848:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668489344 00000100:00100000:0.0:1787785025.013898:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1787785025.013915:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668489344:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785025.013945:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785025.013985:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668489344:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (1634us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785025.013991:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00080000:1.0:1787785025.014049:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787785025.014224:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668489984:0@lo:400:kworker.0 00000100:00100000:2.0:1787785025.014236:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668489600:0@lo:400:kworker.0 00000100:00100000:3.0:1787785025.014323:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668489344:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1787785025.462564:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787785026.313496:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785026.313539:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785026.313562:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785026.313583:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785026.313603:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785026.313619:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785026.313648:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785026.313673:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388480 00000100:00100000:1.0:1787785026.313682:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388480 00000100:00100000:1.0:1787785026.313731:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388608 00000100:00100000:1.0:1787785026.313733:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388608 00000100:00100000:1.0:1787785026.313748:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388864 00000100:00100000:1.0:1787785026.313750:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388864 00000100:00100000:1.0:1787785026.313762:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388992 00000100:00100000:1.0:1787785026.313765:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388992 00000100:00100000:1.0:1787785026.313780:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 598 00000100:00100000:1.0:1787785026.313789:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658388480:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785026.313854:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658388480:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 65us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785026.313862:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 598 00000100:00100000:1.0:1787785026.313880:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:1.0:1787785026.313886:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658388608:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785026.313920:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f7b1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658388608:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785026.313926:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 599 00000100:00100000:1.0:1787785026.313938:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 600 00000100:00100000:1.0:1787785026.313944:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658388864:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785026.313966:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989402a92940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658388864:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (346us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785026.313971:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 600 00000100:00100000:1.0:1787785026.313981:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 601 00000100:00100000:1.0:1787785026.313987:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658388992:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785026.314004:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539128b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658388992:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 18us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785026.314009:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 601 00000100:00080000:1.0:1787785026.314028:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785026.314032:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785026.315295:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388224 00000100:00100000:3.0:1787785026.315307:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388224 00000100:00100000:3.0:1787785026.315354:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 667 00000100:00100000:3.0:1787785026.315362:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658388224:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785026.315409:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658388224:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 48us (1923us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785026.315417:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 667 00000100:00100000:3.0:1787785026.315448:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388352 00000100:00100000:3.0:1787785026.315452:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388352 00000100:00100000:3.0:1787785026.315484:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658388736 00000100:00100000:3.0:1787785026.315487:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658388736 00000100:00100000:3.0:1787785026.315505:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 618 00000100:00100000:3.0:1787785026.315512:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658388352:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785026.315539:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658388352:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (2000us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785026.315544:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 618 00000100:00100000:3.0:1787785026.315555:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 619 00000100:00100000:3.0:1787785026.315560:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658388736:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785026.315580:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b537840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658388736:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (1977us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785026.315586:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 619 00000100:00080000:3.0:1787785026.315602:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:0.0:1787785026.389905:0:26140:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785026.389914:0:26140:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785026.393093:0:26140:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785026.393108:0:26140:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785026.481307:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787785027.510627:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785028.081839:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668490112:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787785028.082366:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785028.082683:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668490112 00000100:00100000:0.0:1787785028.082690:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668490112 00000100:00100000:0.0:1787785028.082760:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:0.0:1787785028.082771:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668490112:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787785028.082807:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785028.082859:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668490112:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785028.082865:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00080000:3.0:1787785028.082951:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785028.083665:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668490112:0@lo:13:osp-pre-0-0.0 00000040:00080000:0.0:1787785028.311043:0:26191:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785028.311053:0:26191:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785028.312708:0:26191:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785028.312724:0:26191:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787785028.529354:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787785029.560420:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785030.129619:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668490496:0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.129716:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785030.129795:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787785030.129802:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668490624:0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.129850:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785030.130047:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668490240:0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.130124:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668490496 00000100:00100000:3.0:1787785030.130133:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668490496 00000100:00100000:3.0:1787785030.130239:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 620 00000100:00100000:3.0:1787785030.130251:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989401ceed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668490496:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1787785030.130406:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785030.130422:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668490624 00000100:00100000:0.0:1787785030.130426:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668490624 00000100:00100000:0.0:1787785030.130480:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:0.0:1787785030.130488:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668490624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.133446:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785030.133558:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668490752:0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.133595:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785030.133628:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1787785030.133636:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989401ceed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668490496:12345-0@lo:400:kworker.0 Request processed in 3384us (3916us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785030.133636:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668490368:0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.133652:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 620 00000100:00100000:3.0:1787785030.133685:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668490752 00000100:00100000:3.0:1787785030.133694:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668490752 00000100:00100000:2.0:1787785030.133715:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785030.133741:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668490880:0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.133748:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668490368 00000100:00100000:3.0:1787785030.133752:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668490368 00000100:00100000:2.0:1787785030.133765:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785030.133789:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:3.0:1787785030.133802:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668490752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.133808:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668490240 00000100:00100000:2.0:1787785030.133812:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668490240 00000100:00100000:2.0:1787785030.133837:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 668 00000100:00100000:2.0:1787785030.133843:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668490240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.133890:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668490240:12345-0@lo:400:kworker.0 Request processed in 47us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785030.133931:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668490752:12345-0@lo:400:kworker.0 Request processed in 131us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785030.133938:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:3.0:1787785030.133951:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:3.0:1787785030.133957:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668490368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.133985:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668490368:12345-0@lo:400:kworker.0 Request processed in 28us (269us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785030.133991:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:2.0:1787785030.134005:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 668 00000100:00100000:3.0:1787785030.134026:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787785030.134039:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668491008:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785030.134056:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668490880 00000100:00100000:2.0:1787785030.134061:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668490880 00000100:00100000:3.0:1787785030.134074:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785030.134098:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 603 00000100:00100000:2.0:1787785030.134107:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668490880:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.134111:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491008 00000100:00100000:3.0:1787785030.134116:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491008 00000100:00100000:3.0:1787785030.134166:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:3.0:1787785030.134174:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d4f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3277:x1874627668491008:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785030.134184:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3278:x1874627668490880:12345-0@lo:400:kworker.0 Request processed in 77us (416us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785030.134192:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 603 00002000:00000024:3.0:1787785030.134209:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785030.134268:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cec780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668490496:0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.134331:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668490368:0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.134337:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668490880:0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.134343:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668490240:0@lo:400:kworker.0 00000100:00100000:2.0:1787785030.134349:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cecb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668490752:0@lo:400:kworker.0 00000100:00100000:3.0:1787785030.134542:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d4f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668491008:12345-0@lo:13:osp-pre-1-0.0 Request processed in 368us (467us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785030.134549:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:0.0:1787785030.134700:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668490624:12345-0@lo:400:kworker.0 Request processed in 4211us (4847us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785030.134709:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00080000:0.0:1787785030.134745:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787785030.134752:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787785030.134754:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785030.134843:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668491008:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785030.134987:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989401cef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668490624:0@lo:400:kworker.0 00000040:00080000:0.0:1787785030.209566:0:26242:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785030.209580:0:26242:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785030.210397:0:26242:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785030.210411:0:26242:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785030.577434:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785031.431455:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785031.431527:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785031.431577:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785031.431605:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785031.431628:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785031.431647:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785031.431678:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785031.431820:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658389120 00000100:00100000:2.0:1787785031.431837:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658389120 00000100:00100000:2.0:1787785031.431916:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00100000:2.0:1787785031.431928:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658389120:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785031.432024:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658389120:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 98us (584us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785031.432033:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00100000:0.0:1787785031.432452:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658389376 00000100:00100000:0.0:1787785031.432459:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658389376 00000100:00100000:0.0:1787785031.432507:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658389888 00000100:00100000:0.0:1787785031.432511:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658389888 00000100:00100000:0.0:1787785031.432532:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658389504 00000100:00100000:0.0:1787785031.432534:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658389504 00000100:00100000:0.0:1787785031.432547:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658389760 00000100:00100000:0.0:1787785031.432549:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658389760 00000100:00100000:0.0:1787785031.432565:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 604 00000100:00100000:0.0:1787785031.432572:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658389376:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785031.432618:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658389632 00000100:00100000:0.0:1787785031.432618:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658389376:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 47us (1091us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785031.432623:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658389632 00000100:00100000:0.0:1787785031.432625:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 604 00000100:00100000:0.0:1787785031.432649:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 605 00000100:00100000:3.0:1787785031.432654:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658389248 00000100:00100000:0.0:1787785031.432655:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658389888:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785031.432656:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658389248 00000100:00100000:3.0:1787785031.432678:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 623 00000100:00100000:0.0:1787785031.432680:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658389888:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (1075us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785031.432685:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658389632:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785031.432686:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 605 00000100:00100000:0.0:1787785031.432697:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 606 00000100:00100000:0.0:1787785031.432703:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658389504:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785031.432715:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658389632:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (1138us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785031.432726:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 623 00000100:00100000:3.0:1787785031.432739:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 624 00000100:00100000:3.0:1787785031.432745:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658389248:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785031.432771:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658389248:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (1142us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785031.432777:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 624 00000100:00100000:0.0:1787785031.432873:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658389504:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (1073us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785031.432881:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 606 00000100:00100000:0.0:1787785031.433093:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 607 00000100:00100000:0.0:1787785031.433100:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658389760:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785031.433130:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658389760:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (1452us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785031.433136:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 607 00000100:00080000:0.0:1787785031.433185:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787785031.433191:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787785031.433194:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787785031.605981:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785032.033633:0:26294:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785032.033647:0:26294:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785032.034221:0:26294:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785032.034233:0:26294:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785032.625845:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785033.201399:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787785033.201429:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668491136:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787785033.201523:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785033.201620:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491136 00000100:00100000:2.0:1787785033.201629:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491136 00000100:00100000:2.0:1787785033.201710:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:2.0:1787785033.201722:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668491136:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785033.201773:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785033.201831:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668491136:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785033.201840:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00080000:3.0:1787785033.202020:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785033.202087:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668491136:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787785033.649370:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785033.821621:0:26345:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785033.821632:0:26345:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785033.825382:0:26345:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785033.825404:0:26345:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:2.0:1787785033.977545:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785033.977555:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785033.978370:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785033.978380:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785034.673379:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785035.250339:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668491264:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785035.250455:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785035.250527:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491264 00000100:00100000:2.0:1787785035.250547:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491264 00000100:00100000:2.0:1787785035.250621:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1787785035.250631:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668491264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785035.250678:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785035.250764:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668491264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 134us (309us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785035.250771:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1787785035.250823:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668491264:0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1787785035.252168:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787785035.253775:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787785035.253811:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668491520:0@lo:400:kworker.0 00000100:00100000:1.0:1787785035.253940:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785035.254043:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668491648:0@lo:400:kworker.0 00000100:00100000:1.0:1787785035.254092:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785035.254140:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491520 00000100:00100000:1.0:1787785035.254180:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491520 00000100:00100000:1.0:1787785035.254239:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491648 00000100:00100000:1.0:1787785035.254242:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491648 00000100:00100000:1.0:1787785035.254276:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:1.0:1787785035.254287:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668491520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785035.254352:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953f6aad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668491520:12345-0@lo:400:kworker.0 Request processed in 66us (411us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785035.254359:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:1.0:1787785035.254380:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:1.0:1787785035.254388:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668491648:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785035.254416:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940304ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668491648:12345-0@lo:400:kworker.0 Request processed in 29us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785035.254423:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00080000:1.0:1787785035.254442:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787785035.254479:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668491648:0@lo:400:kworker.0 00000100:00100000:1.0:1787785035.254493:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668491520:0@lo:400:kworker.0 00000100:00100000:0.0:1787785035.254720:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668491776:0@lo:400:kworker.0 00000100:00100000:0.0:1787785035.254765:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785035.254807:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1787785035.254815:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953912ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668491392:0@lo:400:kworker.0 00000100:00100000:0.0:1787785035.254871:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785035.254909:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668491904:0@lo:400:kworker.0 00000100:00100000:0.0:1787785035.254935:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785035.254965:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491392 00000100:00100000:0.0:1787785035.254969:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491392 00000100:00100000:0.0:1787785035.255004:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:0.0:1787785035.255012:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668491392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785035.255051:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668491392:12345-0@lo:400:kworker.0 Request processed in 40us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785035.255058:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:0.0:1787785035.255110:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491904 00000100:00100000:0.0:1787785035.255114:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491904 00000100:00100000:0.0:1787785035.255172:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 627 00000100:00100000:0.0:1787785035.255179:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668491904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785035.255211:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668491904:12345-0@lo:400:kworker.0 Request processed in 33us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785035.255217:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 627 00000100:00080000:0.0:1787785035.255237:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785035.255255:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953912ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668491392:0@lo:400:kworker.0 00000100:00100000:0.0:1787785035.255265:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539129680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668491904:0@lo:400:kworker.0 00000100:00100000:1.0:1787785035.256412:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668491776 00000100:00100000:1.0:1787785035.256423:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668491776 00000100:00100000:1.0:1787785035.256496:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 608 00000100:00100000:1.0:1787785035.256508:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668491776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785035.256599:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668491776:12345-0@lo:400:kworker.0 Request processed in 94us (1833us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785035.256607:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 608 00000100:00100000:3.0:1787785035.256698:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668491776:0@lo:400:kworker.0 00000100:00080000:0.0:1787785035.256814:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787785035.697308:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785035.775967:0:26396:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785035.775976:0:26396:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785035.776915:0:26396:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785035.776927:0:26396:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785036.551618:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785036.551662:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785036.551756:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785036.551816:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390144 00000100:00100000:0.0:1787785036.551840:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390144 00000100:00100000:0.0:1787785036.551923:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 628 00000100:00100000:0.0:1787785036.551975:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658390144:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785036.552054:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658390144:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 122us (447us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785036.552061:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 628 00000100:00080000:2.0:1787785036.552171:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785036.555870:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785036.555899:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785036.556011:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390400 00000100:00100000:1.0:1787785036.556021:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390400 00000100:00100000:2.0:1787785036.556261:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390016 00000100:00100000:2.0:1787785036.556266:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390016 00000100:00100000:2.0:1787785036.556314:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 671 00000100:00100000:2.0:1787785036.556325:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658390016:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785036.556396:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658390016:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 73us (535us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785036.556405:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 671 00000100:00100000:1.0:1787785036.556743:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390656 00000100:00100000:1.0:1787785036.556748:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390656 00000100:00100000:1.0:1787785036.556842:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 609 00000100:00100000:1.0:1787785036.556851:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658390400:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785036.556888:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658390400:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 39us (5225us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785036.556895:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 609 00000100:00100000:1.0:1787785036.556907:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 610 00000100:00100000:1.0:1787785036.556913:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658390656:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785036.556936:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658390656:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (5182us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785036.556942:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 610 00000100:00100000:3.0:1787785036.557222:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785036.557263:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785036.557282:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390272 00000100:00100000:3.0:1787785036.557285:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390272 00000100:00100000:3.0:1787785036.557309:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390784 00000100:00100000:3.0:1787785036.557312:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390784 00000100:00100000:3.0:1787785036.557328:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 611 00000100:00100000:3.0:1787785036.557334:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658390272:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785036.557366:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658390272:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (1467us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785036.557372:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 611 00000100:00100000:3.0:1787785036.557382:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 612 00000100:00100000:3.0:1787785036.557388:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658390784:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785036.557409:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658390784:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (147us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785036.557414:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 612 00000100:00100000:3.0:1787785036.557439:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390528 00000100:00100000:3.0:1787785036.557442:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390528 00000100:00100000:3.0:1787785036.557643:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 629 00000100:00100000:3.0:1787785036.557650:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658390528:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785036.557678:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658390528:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (458us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785036.557685:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 629 00000100:00080000:2.0:1787785036.557840:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787785036.557845:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787785036.557847:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787785036.721638:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785037.643613:0:26447:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785037.643624:0:26447:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785037.647735:0:26447:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785037.647756:0:26447:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785037.746251:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785038.321500:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787785038.321525:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668492032:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787785038.321593:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785038.321792:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492032 00000100:00100000:2.0:1787785038.321800:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492032 00000100:00100000:2.0:1787785038.321884:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:2.0:1787785038.321893:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668492032:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785038.321926:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785038.321969:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668492032:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (376us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785038.321975:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00080000:0.0:1787785038.322420:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785038.322512:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668492032:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787785038.770141:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785039.362190:0:26499:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785039.362200:0:26499:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785039.362752:0:26499:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785039.362762:0:26499:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785039.793459:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785040.369991:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668492160:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785040.370069:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785040.370884:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989537040000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668492672:0@lo:400:kworker.0 00000100:00100000:3.0:1787785040.370922:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785040.370951:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787785040.370957:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895370430c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668492544:0@lo:400:kworker.0 00000100:00100000:3.0:1787785040.370983:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785040.371012:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492672 00000100:00100000:3.0:1787785040.371018:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492672 00000100:00100000:3.0:1787785040.371070:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 613 00000100:00100000:3.0:1787785040.371080:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895370412c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668492672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785040.371128:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895370412c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668492672:12345-0@lo:400:kworker.0 Request processed in 50us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785040.371135:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 613 00000100:00100000:3.0:1787785040.371179:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492544 00000100:00100000:3.0:1787785040.371183:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492544 00000100:00100000:3.0:1787785040.371208:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:3.0:1787785040.371215:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895370403c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668492544:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785040.371246:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895370403c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668492544:12345-0@lo:400:kworker.0 Request processed in 32us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785040.371252:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:3.0:1787785040.371289:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895370430c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668492544:0@lo:400:kworker.0 00000100:00100000:3.0:1787785040.371300:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989537040000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668492672:0@lo:400:kworker.0 00000100:00100000:1.0:1787785040.371565:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989537040780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668492416:0@lo:400:kworker.0 00000100:00100000:1.0:1787785040.371603:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785040.371635:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1787785040.371641:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668492288:0@lo:400:kworker.0 00000100:00100000:1.0:1787785040.371671:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785040.371693:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989537043480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668492800:0@lo:400:kworker.0 00000100:00100000:1.0:1787785040.371714:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785040.371745:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492416 00000100:00100000:1.0:1787785040.371750:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492416 00000100:00100000:1.0:1787785040.371776:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492800 00000100:00100000:1.0:1787785040.371779:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492800 00000100:00100000:1.0:1787785040.371815:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:1.0:1787785040.371822:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668492416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785040.371860:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539129680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668492416:12345-0@lo:400:kworker.0 Request processed in 38us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785040.371866:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00100000:1.0:1787785040.371879:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1787785040.371885:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668492800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785040.371911:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953912ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668492800:12345-0@lo:400:kworker.0 Request processed in 27us (196us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785040.371917:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1787785040.371947:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989537043480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668492800:0@lo:400:kworker.0 00000100:00100000:1.0:1787785040.371957:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989537040780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668492416:0@lo:400:kworker.0 00000100:00080000:0.0:1787785040.372106:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787785040.372111:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785040.372138:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492288 00000100:00100000:0.0:1787785040.372142:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492288 00000100:00100000:0.0:1787785040.372188:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:0.0:1787785040.372195:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668492288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785040.372235:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668492288:12345-0@lo:400:kworker.0 Request processed in 41us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785040.372242:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00100000:0.0:1787785040.372276:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668492288:0@lo:400:kworker.0 00000100:00100000:2.0:1787785040.373008:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492160 00000100:00100000:2.0:1787785040.373016:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492160 00000100:00100000:2.0:1787785040.373074:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:2.0:1787785040.373083:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668492160:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785040.373114:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785040.373204:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668492160:12345-0@lo:13:osp-pre-1-0.0 Request processed in 123us (3135us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785040.373211:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00080000:0.0:1787785040.373473:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787785040.373494:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668492160:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1787785040.818515:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785041.269638:0:26550:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785041.269647:0:26550:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785041.270700:0:26550:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785041.270714:0:26550:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787785041.678322:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785041.678372:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785041.678421:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785041.678461:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785041.678498:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785041.678520:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785041.678522:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391296 00000100:00100000:0.0:1787785041.678531:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391296 00000100:00100000:1.0:1787785041.678558:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785041.678584:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391552 00000100:00100000:0.0:1787785041.678586:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391552 00000100:00100000:1.0:1787785041.678592:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391680 00000100:00100000:1.0:1787785041.678595:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391680 00000100:00100000:0.0:1787785041.678606:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391168 00000100:00100000:0.0:1787785041.678609:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391168 00000100:00100000:1.0:1787785041.678616:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 614 00000100:00100000:3.0:1787785041.678723:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391040 00000100:00100000:3.0:1787785041.678727:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391040 00000100:00100000:3.0:1787785041.678755:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391424 00000100:00100000:3.0:1787785041.678758:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391424 00000100:00100000:3.0:1787785041.678777:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00100000:3.0:1787785041.678796:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658391040:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785041.678882:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894040092c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2356:x1874627658391040:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 95us (569us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785041.678889:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00100000:3.0:1787785041.678906:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 634 00000100:00100000:3.0:1787785041.678913:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536ebb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658391424:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785041.678939:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536ebb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658391424:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (380us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785041.678945:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 634 00000100:00100000:1.0:1787785041.679088:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658391296:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785041.679127:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98940400a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658391296:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 503us (754us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785041.680513:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 615 00000100:00100000:0.0:1787785041.680532:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536ebb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658391552:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785041.680585:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536ebb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658391552:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 61us (2164us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785041.680594:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 615 00000100:00100000:0.0:1787785041.680618:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 616 00000100:00100000:0.0:1787785041.680626:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527deb850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658391680:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785041.680651:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527deb850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658391680:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (2190us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785041.680657:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 616 00000100:00100000:0.0:1787785041.680670:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 617 00000100:00100000:0.0:1787785041.680677:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658391168:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785041.680699:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658391168:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (2178us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785041.680705:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 617 00000100:00100000:0.0:1787785041.681042:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658390912 00000100:00100000:0.0:1787785041.681049:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658390912 00000100:00100000:0.0:1787785041.681095:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00100000:0.0:1787785041.681102:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658390912:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785041.681130:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7ce850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658390912:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (2631us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785041.681136:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00080000:2.0:1787785041.682426:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787785041.682436:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787785041.682439:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787785041.685935:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 614 00010000:00010000:3.0:1787785041.841390:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787785042.868954:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785043.190316:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785043.190327:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785043.192365:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785043.192388:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:2.0:1787785043.192750:0:26601:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785043.192755:0:26601:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785043.195445:0:26601:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785043.195461:0:26601:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785043.441524:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668492928:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787785043.441594:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785043.441798:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668492928 00000100:00100000:2.0:1787785043.441806:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668492928 00000100:00100000:2.0:1787785043.441871:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:2.0:1787785043.441880:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668492928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785043.441917:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785043.441965:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668492928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785043.441973:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00080000:1.0:1787785043.442040:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785043.442105:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668492928:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787785043.892731:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787785044.915531:0:482:0:(ldlm_pool.c:1330:ldlm_pools_recalc_task()) Negative interval(0) 00010000:00010000:0.0:1787785044.921107:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785045.071915:0:26653:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785045.071931:0:26653:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785045.074975:0:26653:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785045.075009:0:26653:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785045.489777:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787785045.489808:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668493056:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785045.489914:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785045.490259:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031261c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668493440:0@lo:400:kworker.0 00000100:00100000:0.0:1787785045.490301:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785045.490332:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787785045.490447:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493440 00000100:00100000:3.0:1787785045.490457:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493440 00000100:00100000:3.0:1787785045.490527:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:3.0:1787785045.490540:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668493440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785045.490600:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668493440:12345-0@lo:400:kworker.0 Request processed in 63us (297us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785045.490608:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00080000:3.0:1787785045.490634:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785045.490656:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668493184:0@lo:400:kworker.0 00000100:00100000:3.0:1787785045.490696:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785045.490720:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668493696:0@lo:400:kworker.0 00000100:00100000:3.0:1787785045.490745:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785045.490774:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787785045.490781:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668493568:0@lo:400:kworker.0 00000100:00100000:3.0:1787785045.490808:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785045.490882:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668493312:0@lo:400:kworker.0 00000100:00100000:0.0:1787785045.490908:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785045.491013:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031261c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668493440:0@lo:400:kworker.0 00000100:00100000:0.0:1787785045.491048:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493696 00000100:00100000:0.0:1787785045.491051:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493696 00000100:00100000:2.0:1787785045.491175:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493056 00000100:00100000:2.0:1787785045.491180:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493056 00000100:00100000:0.0:1787785045.491776:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493312 00000100:00100000:0.0:1787785045.491782:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493312 00000100:00100000:0.0:1787785045.491825:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:0.0:1787785045.491834:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668493312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785045.491884:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668493312:12345-0@lo:400:kworker.0 Request processed in 51us (974us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785045.491890:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:0.0:1787785045.491915:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493184 00000100:00100000:0.0:1787785045.491918:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493184 00000100:00100000:0.0:1787785045.491951:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:0.0:1787785045.491958:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668493184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785045.491986:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951c7cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668493184:12345-0@lo:400:kworker.0 Request processed in 29us (1290us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785045.491992:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:0.0:1787785045.492040:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668493312:0@lo:400:kworker.0 00000100:00100000:2.0:1787785045.492264:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:2.0:1787785045.492275:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668493056:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785045.492312:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785045.492367:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668493056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (2455us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785045.492374:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00000100:00080000:2.0:1787785045.492404:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787785045.492409:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787785045.493080:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1787785045.493091:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895370430c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668493696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785045.493134:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895370430c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668493696:12345-0@lo:400:kworker.0 Request processed in 47us (2389us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785045.493141:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:3.0:1787785045.493535:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668493184:0@lo:400:kworker.0 00000100:00100000:3.0:1787785045.493550:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668493696:0@lo:400:kworker.0 00000100:00100000:3.0:1787785045.493568:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668493056:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787785045.493626:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493568 00000100:00100000:0.0:1787785045.493633:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493568 00000100:00100000:0.0:1787785045.493719:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 618 00000100:00100000:0.0:1787785045.493728:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895370421c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668493568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785045.494104:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895370421c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668493568:12345-0@lo:400:kworker.0 Request processed in 376us (3293us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785045.494113:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 618 00000100:00080000:3.0:1787785045.494180:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785045.494254:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668493568:0@lo:400:kworker.0 00010000:00010000:1.0:1787785045.938533:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785046.799477:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785046.799530:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785046.799560:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785046.799570:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391808 00000100:00100000:2.0:1787785046.799581:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785046.799585:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391808 00000100:00100000:2.0:1787785046.799606:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785046.799624:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785046.799649:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 675 00000100:00100000:2.0:1787785046.799660:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785046.799661:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658391808:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785046.799685:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392064 00000100:00100000:2.0:1787785046.799689:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392064 00000100:00100000:2.0:1787785046.799730:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392576 00000100:00100000:2.0:1787785046.799733:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392576 00000100:00100000:2.0:1787785046.799763:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392192 00000100:00100000:2.0:1787785046.799766:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392192 00000100:00100000:2.0:1787785046.799780:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392448 00000100:00100000:2.0:1787785046.799782:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392448 00000100:00100000:2.0:1787785046.799798:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 619 00000100:00100000:2.0:1787785046.799805:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658392064:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785046.799834:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658391808:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 174us (367us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785046.799836:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658392064:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (306us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785046.799844:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 675 00000100:00100000:2.0:1787785046.799844:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 619 00000100:00100000:2.0:1787785046.799861:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 620 00000100:00100000:2.0:1787785046.799868:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658392576:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785046.799894:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658392576:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785046.799901:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 620 00000100:00100000:2.0:1787785046.799912:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 621 00000100:00100000:2.0:1787785046.799919:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658392192:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785046.799943:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658392192:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785046.799949:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 621 00000100:00100000:2.0:1787785046.799961:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 622 00000100:00100000:2.0:1787785046.799966:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658392448:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785046.799989:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658392448:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785046.799994:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 622 00000100:00100000:3.0:1787785046.800012:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392320 00000100:00100000:3.0:1787785046.800016:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392320 00000100:00100000:3.0:1787785046.800100:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658391936 00000100:00100000:3.0:1787785046.800104:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658391936 00000100:00100000:3.0:1787785046.800130:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 638 00000100:00100000:3.0:1787785046.800136:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658392320:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785046.800194:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658392320:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 58us (633us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785046.800202:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 638 00000100:00100000:3.0:1787785046.800213:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00100000:3.0:1787785046.800219:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658391936:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785046.800853:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658391936:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 630us (1242us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785046.800863:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00080000:3.0:1787785046.801030:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787785046.801036:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787785046.801039:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:2.0:1787785046.934487:0:26704:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785046.934504:0:26704:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785046.935056:0:26704:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785046.935068:0:26704:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785046.964708:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787785047.985403:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787785048.562627:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668493824:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787785048.562731:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785048.562864:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493824 00000100:00100000:1.0:1787785048.562874:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493824 00000100:00100000:1.0:1787785048.562963:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:1.0:1787785048.562976:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668493824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787785048.563025:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787785048.567107:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668493824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4120us (4361us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785048.567126:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00080000:1.0:1787785048.567188:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785048.573622:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668493824:0@lo:13:osp-pre-0-0.0 00000040:00080000:2.0:1787785048.974040:0:26755:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785048.974054:0:26755:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785048.985295:0:26755:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785048.985384:0:26755:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787785049.010430:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787785050.033516:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787785050.611770:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668494336:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.611898:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785050.611969:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1787785050.611978:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668493952:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787785050.612012:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785050.612033:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668494464:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612060:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785050.612102:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668494208:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612128:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785050.612171:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668494720:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612203:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785050.612225:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787785050.612231:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540666580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668494080:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612265:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785050.612288:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031261c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668494592:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612313:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785050.612351:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494464 00000100:00100000:0.0:1787785050.612359:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494464 00000100:00100000:0.0:1787785050.612430:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494720 00000100:00100000:0.0:1787785050.612435:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494720 00000100:00100000:0.0:1787785050.612545:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:0.0:1787785050.612557:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668494464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612622:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668494464:12345-0@lo:400:kworker.0 Request processed in 67us (559us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785050.612630:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00000100:00100000:0.0:1787785050.612657:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:0.0:1787785050.612664:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668494720:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612698:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895063361c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668494720:12345-0@lo:400:kworker.0 Request processed in 35us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785050.612703:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:0.0:1787785050.612736:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494336 00000100:00100000:0.0:1787785050.612741:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494336 00000100:00100000:0.0:1787785050.612773:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494208 00000100:00100000:0.0:1787785050.612776:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494208 00000100:00100000:0.0:1787785050.612794:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494592 00000100:00100000:0.0:1787785050.612796:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494592 00000100:00100000:0.0:1787785050.612815:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:0.0:1787785050.612822:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668494336:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612858:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668494336:12345-0@lo:400:kworker.0 Request processed in 37us (962us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785050.612865:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00100000:0.0:1787785050.612881:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:0.0:1787785050.612888:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668494208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612915:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668494208:12345-0@lo:400:kworker.0 Request processed in 28us (785us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785050.612921:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:0.0:1787785050.612933:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:0.0:1787785050.612939:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668494592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.612973:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668494592:12345-0@lo:400:kworker.0 Request processed in 34us (659us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785050.612979:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:0.0:1787785050.613009:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494080 00000100:00100000:0.0:1787785050.613013:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494080 00000100:00100000:0.0:1787785050.613045:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:0.0:1787785050.613052:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668494080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.613085:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668494080:12345-0@lo:400:kworker.0 Request processed in 34us (819us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785050.613091:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:0.0:1787785050.613191:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540666580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668494080:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.613209:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031261c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668494592:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.613223:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668494208:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.613237:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668494720:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.613287:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403127c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668494464:0@lo:400:kworker.0 00000100:00100000:0.0:1787785050.613302:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031252c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668494336:0@lo:400:kworker.0 00000100:00100000:1.0:1787785050.617075:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668493952 00000100:00100000:1.0:1787785050.617094:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668493952 00000100:00100000:1.0:1787785050.617200:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:1.0:1787785050.617213:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668493952:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787785050.617266:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787785050.617386:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540289a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668493952:12345-0@lo:13:osp-pre-1-0.0 Request processed in 175us (5372us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785050.617394:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00080000:1.0:1787785050.617422:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787785050.617429:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785050.617432:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785050.618678:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668493952:0@lo:13:osp-pre-1-0.0 00000040:00080000:0.0:1787785050.993870:0:26806:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785050.993879:0:26806:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785050.994839:0:26806:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785050.994852:0:26806:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785051.057372:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785051.910881:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785051.910936:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785051.910968:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392832 00000100:00100000:0.0:1787785051.910978:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392832 00000100:00100000:3.0:1787785051.910994:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785051.911039:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785051.911068:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785051.911074:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:0.0:1787785051.911088:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537040b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658392832:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785051.911097:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785051.911300:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537040b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658392832:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 214us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785051.911309:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:1.0:1787785051.911414:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393088 00000100:00100000:1.0:1787785051.911419:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393088 00000100:00100000:1.0:1787785051.911450:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393344 00000100:00100000:1.0:1787785051.911453:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393344 00000100:00100000:1.0:1787785051.911469:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392960 00000100:00100000:1.0:1787785051.911471:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392960 00000100:00100000:1.0:1787785051.911491:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 625 00000100:00100000:1.0:1787785051.911498:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537042940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658393088:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785051.911537:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393216 00000100:00100000:0.0:1787785051.911541:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393216 00000100:00100000:0.0:1787785051.911563:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 644 00000100:00100000:0.0:1787785051.911568:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537040f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658393216:12345-192.168.203.43@tcp:400:kworker.0 00000100:00080000:2.0:1787785051.911702:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787785051.911707:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787785051.911709:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787785051.911731:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 626 00000100:00100000:2.0:1787785051.911740:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537043840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658393344:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785051.913237:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785051.913365:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658392704 00000100:00100000:3.0:1787785051.913373:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658392704 00000100:00100000:3.0:1787785051.913423:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 677 00000100:00100000:3.0:1787785051.913433:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658392704:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785051.914584:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537042940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658393088:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 3078us (3650us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785051.914595:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 625 00000100:00100000:1.0:1787785051.914622:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393472 00000100:00100000:1.0:1787785051.914628:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393472 00000100:00100000:1.0:1787785051.914667:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 627 00000100:00100000:1.0:1787785051.914674:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537041e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658392960:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785051.914708:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537041e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658392960:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 35us (3640us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785051.914713:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 627 00000100:00100000:1.0:1787785051.914736:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 628 00000100:00100000:1.0:1787785051.914743:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537041680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658393472:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785051.914763:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537041680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658393472:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (1534us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785051.914768:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 628 00000100:00100000:0.0:1787785051.915126:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537040f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658393216:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 3554us (4027us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785051.915320:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 644 00000100:00100000:2.0:1787785051.915451:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537043840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658393344:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 3711us (4468us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785051.915460:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 626 00000100:00080000:0.0:1787785051.916002:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787785051.916638:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658392704:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 3200us (5592us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785051.916649:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 677 00010000:00010000:1.0:1787785052.081345:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785052.735914:0:26858:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785052.735923:0:26858:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785052.739704:0:26858:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785052.739732:0:26858:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785053.106806:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785053.427403:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785053.427413:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785053.427988:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785053.428004:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785053.681498:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668494848:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787785053.681585:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785053.681925:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494848 00000100:00100000:2.0:1787785053.681934:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494848 00000100:00100000:2.0:1787785053.682002:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:2.0:1787785053.682013:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668494848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785053.682046:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785053.682109:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668494848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785053.682117:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:3.0:1787785053.682194:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668494848:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787785053.682364:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787785054.129648:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785054.575275:0:26909:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785054.575285:0:26909:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785054.577381:0:26909:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785054.577399:0:26909:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787785055.153536:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787785055.729913:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787785055.729937:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668494976:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787785055.730007:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785055.730185:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668494976 00000100:00100000:3.0:1787785055.730193:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668494976 00000100:00100000:3.0:1787785055.730252:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:3.0:1787785055.730261:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536d3e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668494976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787785055.730305:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787785055.730350:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536d3e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668494976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785055.730356:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00080000:0.0:1787785055.730678:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787785055.730716:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668495232:0@lo:400:kworker.0 00000100:00100000:0.0:1787785055.730765:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785055.730792:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787785055.730799:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668495104:0@lo:400:kworker.0 00000100:00100000:0.0:1787785055.730827:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785055.730850:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668495616:0@lo:400:kworker.0 00000100:00100000:0.0:1787785055.730871:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785055.730901:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668495232 00000100:00100000:0.0:1787785055.730904:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668495232 00000100:00100000:0.0:1787785055.730934:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668495616 00000100:00100000:0.0:1787785055.730937:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668495616 00000100:00100000:0.0:1787785055.730960:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:0.0:1787785055.730966:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668495232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785055.731141:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668495232:12345-0@lo:400:kworker.0 Request processed in 174us (375us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785055.731172:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:0.0:1787785055.731192:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1787785055.731198:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668495616:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785055.731227:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668495616:12345-0@lo:400:kworker.0 Request processed in 30us (356us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785055.731233:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1787785055.731251:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668495104 00000100:00100000:0.0:1787785055.731255:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668495104 00000100:00100000:0.0:1787785055.731277:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 678 00000100:00100000:0.0:1787785055.731284:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668495104:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785055.731313:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668495104:12345-0@lo:400:kworker.0 Request processed in 30us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785055.731319:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 678 00000100:00080000:0.0:1787785055.731350:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785055.731362:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668495360:0@lo:400:kworker.0 00000100:00100000:0.0:1787785055.731385:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785055.731511:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668495488:0@lo:400:kworker.0 00000100:00100000:3.0:1787785055.731541:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785055.731576:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668495488 00000100:00100000:3.0:1787785055.731580:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668495488 00000100:00100000:3.0:1787785055.731614:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:3.0:1787785055.731620:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668495488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785055.731654:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537040000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668495488:12345-0@lo:400:kworker.0 Request processed in 35us (112us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785055.731660:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00100000:3.0:1787785055.731700:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668495488:0@lo:400:kworker.0 00000100:00100000:3.0:1787785055.731738:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668495360 00000100:00080000:2.0:1787785055.732457:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785055.732499:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668495104:0@lo:400:kworker.0 00000100:00100000:2.0:1787785055.732512:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668495616:0@lo:400:kworker.0 00000100:00100000:2.0:1787785055.732522:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668495232:0@lo:400:kworker.0 00000100:00100000:3.0:1787785055.732644:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668495360 00000100:00080000:2.0:1787785055.732725:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785055.732798:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668494976:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785055.733509:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:3.0:1787785055.733522:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668495360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785055.733596:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668495360:12345-0@lo:400:kworker.0 Request processed in 77us (2209us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785055.733603:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:0.0:1787785055.734113:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668495360:0@lo:400:kworker.0 00010000:00010000:0.0:1787785056.183243:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785056.419998:0:26960:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785056.420007:0:26960:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785056.420648:0:26960:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785056.420663:0:26960:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785057.032587:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785057.032640:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785057.032673:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785057.032729:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785057.032780:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785057.032799:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785057.032818:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785057.032853:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393600 00000100:00100000:3.0:1787785057.032862:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393600 00000100:00100000:3.0:1787785057.032907:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 679 00000100:00100000:3.0:1787785057.032917:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658393600:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785057.032984:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658393600:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 67us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785057.032994:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 679 00000100:00100000:0.0:1787785057.033099:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393728 00000100:00100000:0.0:1787785057.033105:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393728 00000100:00100000:0.0:1787785057.033174:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658394112 00000100:00100000:0.0:1787785057.033178:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658394112 00000100:00100000:0.0:1787785057.033201:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 648 00000100:00100000:0.0:1787785057.033208:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537042d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658393728:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785057.033241:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537042d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658393728:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 33us (665us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785057.033247:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 648 00000100:00100000:0.0:1787785057.033264:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00100000:0.0:1787785057.033270:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895370412c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658394112:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785057.033292:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895370412c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658394112:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785057.033298:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00100000:0.0:1787785057.033315:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393984 00000100:00100000:0.0:1787785057.033318:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393984 00000100:00100000:0.0:1787785057.033342:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658394240 00000100:00100000:0.0:1787785057.033345:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658394240 00000100:00100000:0.0:1787785057.033361:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658393856 00000100:00100000:0.0:1787785057.033364:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658393856 00000100:00100000:0.0:1787785057.033375:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658394368 00000100:00100000:0.0:1787785057.033377:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658394368 00000100:00100000:0.0:1787785057.033389:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 630 00000100:00100000:0.0:1787785057.033395:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537042580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658393984:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785057.033418:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537042580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658393984:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (778us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785057.033423:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 630 00000100:00100000:0.0:1787785057.033433:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 631 00000100:00100000:0.0:1787785057.033438:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537041a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658394240:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785057.033455:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537041a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658394240:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 17us (782us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785057.033460:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 631 00000100:00100000:0.0:1787785057.033469:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 632 00000100:00100000:0.0:1787785057.033475:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537043c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658393856:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785057.033494:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537043c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658393856:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (713us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785057.033499:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 632 00000100:00100000:0.0:1787785057.033510:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00100000:0.0:1787785057.033515:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537040780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658394368:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785057.033536:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537040780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658394368:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (717us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785057.033542:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 633 00000100:00080000:3.0:1787785057.033705:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787785057.033710:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787785057.033713:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787785057.202741:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787785058.225433:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785058.467494:0:27011:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785058.467508:0:27011:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785058.468556:0:27011:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785058.468574:0:27011:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785058.801550:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787785058.801585:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668495744:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787785058.801884:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785058.802026:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668495744 00000100:00100000:2.0:1787785058.802038:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668495744 00000100:00100000:2.0:1787785058.802134:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:2.0:1787785058.802183:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989537043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668495744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785058.802245:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785058.802317:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989537043480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668495744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 173us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785058.802326:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1787785058.802411:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668495744:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787785058.804014:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787785059.249345:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787785060.282996:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785060.394709:0:27063:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785060.394719:0:27063:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785060.395791:0:27063:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785060.395809:0:27063:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1787785060.850628:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506336940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668495872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785060.850717:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785060.850717:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668496256:0@lo:400:kworker.0 00000100:00100000:0.0:1787785060.850763:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785060.850791:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787785060.850792:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668496384:0@lo:400:kworker.0 00000100:00100000:0.0:1787785060.850800:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506337c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668496128:0@lo:400:kworker.0 00000100:00100000:2.0:1787785060.850830:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785060.850837:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785060.850863:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1787785060.850869:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668496000:0@lo:400:kworker.0 00000100:00100000:3.0:1787785060.851006:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496256 00000100:00100000:3.0:1787785060.851014:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496256 00000100:00100000:3.0:1787785060.851068:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496128 00000100:00100000:3.0:1787785060.851072:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496128 00000100:00100000:3.0:1787785060.851094:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:3.0:1787785060.851186:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668496256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785060.851211:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668496640:0@lo:400:kworker.0 00000100:00100000:1.0:1787785060.851303:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:1.0:1787785060.851313:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668496128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785060.863565:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785060.863628:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031270c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668496512:0@lo:400:kworker.0 00000100:00100000:2.0:1787785060.863684:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785060.863730:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668495872 00000100:00100000:2.0:1787785060.863738:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668495872 00000100:00100000:2.0:1787785060.863790:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:2.0:1787785060.863799:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668495872:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785060.863836:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785060.863880:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668495872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (13166us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785060.863887:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:2.0:1787785060.863912:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496384 00000100:00100000:2.0:1787785060.863916:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496384 00000100:00100000:2.0:1787785060.863945:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1787785060.863951:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668496384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785060.863985:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668496384:12345-0@lo:400:kworker.0 Request processed in 34us (13155us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785060.863993:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1787785060.864014:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496000 00000100:00100000:3.0:1787785060.864076:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403124f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668496256:12345-0@lo:400:kworker.0 Request processed in 12892us (13312us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785060.864085:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:3.0:1787785060.864099:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496512 00000100:00100000:3.0:1787785060.864102:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496512 00000100:00100000:3.0:1787785060.864119:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:3.0:1787785060.864125:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668496512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785060.864170:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668496512:12345-0@lo:400:kworker.0 Request processed in 45us (486us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785060.864177:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00080000:3.0:1787785060.864194:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787785060.864199:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787785060.864202:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785060.864277:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031270c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668496512:0@lo:400:kworker.0 00000100:00100000:3.0:1787785060.864289:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894030492c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668496384:0@lo:400:kworker.0 00000100:00100000:3.0:1787785060.864307:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506336940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668495872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785060.864413:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496000 00000100:00100000:2.0:1787785060.864441:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 680 00000100:00100000:2.0:1787785060.864447:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668496000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785060.864483:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668496000:12345-0@lo:400:kworker.0 Request processed in 36us (927us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785060.864489:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 680 00000100:00100000:1.0:1787785060.864558:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031252c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668496128:12345-0@lo:400:kworker.0 Request processed in 13245us (13719us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785060.864568:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00100000:3.0:1787785060.864647:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506334780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668496000:0@lo:400:kworker.0 00000100:00100000:0.0:1787785060.867201:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785060.867322:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506335a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668496256:0@lo:400:kworker.0 00000100:00100000:0.0:1787785060.867349:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989506337c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668496128:0@lo:400:kworker.0 00000100:00100000:1.0:1787785060.870723:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496640 00000100:00100000:1.0:1787785060.870737:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496640 00000100:00100000:1.0:1787785060.870798:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:1.0:1787785060.870808:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668496640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785060.871543:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403127c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668496640:12345-0@lo:400:kworker.0 Request processed in 732us (4344us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785060.871553:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:2.0:1787785060.871700:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668496640:0@lo:400:kworker.0 00010000:00010000:1.0:1787785061.297431:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785062.151927:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785062.151970:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785062.152005:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785062.152040:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785062.152063:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785062.152085:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785062.152109:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785062.152137:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658394752 00000100:00100000:2.0:1787785062.152173:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658394752 00000100:00100000:2.0:1787785062.152227:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395264 00000100:00100000:2.0:1787785062.152231:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395264 00000100:00100000:2.0:1787785062.152247:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658394880 00000100:00100000:2.0:1787785062.152249:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658394880 00000100:00100000:0.0:1787785062.152357:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658394496 00000100:00100000:0.0:1787785062.152362:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658394496 00000100:00100000:0.0:1787785062.152391:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 681 00000100:00100000:0.0:1787785062.152401:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658394496:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785062.152478:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658394496:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (558us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785062.152485:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 681 00000100:00100000:2.0:1787785062.152628:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395136 00000100:00100000:2.0:1787785062.152632:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395136 00000100:00100000:2.0:1787785062.152759:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 636 00000100:00100000:2.0:1787785062.152766:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658394752:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785062.152796:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658394752:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 31us (826us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785062.152802:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 636 00000100:00100000:2.0:1787785062.152816:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 637 00000100:00100000:2.0:1787785062.152822:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658395264:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785062.152844:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658395264:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (805us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785062.152851:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 637 00000100:00100000:2.0:1787785062.152861:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 638 00000100:00100000:2.0:1787785062.152867:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658394880:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785062.152887:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658394880:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (803us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785062.152893:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 638 00000100:00100000:2.0:1787785062.152902:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00100000:2.0:1787785062.152909:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658395136:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785062.152927:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658395136:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 19us (818us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785062.152932:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 639 00000100:00100000:2.0:1787785062.152950:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395008 00000100:00100000:2.0:1787785062.152953:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395008 00000100:00100000:2.0:1787785062.152985:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658394624 00000100:00100000:2.0:1787785062.152988:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658394624 00000100:00100000:2.0:1787785062.153004:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 653 00000100:00100000:2.0:1787785062.153009:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658395008:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785062.153032:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658395008:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (1027us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785062.153039:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 653 00000100:00100000:2.0:1787785062.153048:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 654 00000100:00100000:2.0:1787785062.153054:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536e9cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658394624:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785062.153071:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536e9cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658394624:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 18us (1008us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785062.153077:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 654 00000100:00080000:2.0:1787785062.153100:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787785062.153105:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787785062.153107:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787785062.327369:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785062.361458:0:27114:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785062.361468:0:27114:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785062.367834:0:27114:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785062.367854:0:27114:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785063.346204:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785063.666098:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785063.666109:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785063.667105:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785063.667126:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785063.921655:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668496768:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787785063.921728:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785063.921986:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496768 00000100:00100000:2.0:1787785063.921994:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496768 00000100:00100000:2.0:1787785063.922136:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1787785063.922178:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668496768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785063.922224:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785063.922280:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031261c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668496768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 135us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785063.922288:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:1.0:1787785063.922385:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668496768:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787785063.922677:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:0.0:1787785064.126247:0:27165:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785064.126257:0:27165:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785064.126808:0:27165:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785064.126818:0:27165:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787785064.369435:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787785065.396042:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787785065.969359:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894031270c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668497024:0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.969373:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668496896:0@lo:400:kworker.0 00000100:00100000:1.0:1787785065.969442:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668497152:0@lo:400:kworker.0 00000100:00100000:0.0:1787785065.972014:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785065.972087:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668497536:0@lo:400:kworker.0 00000100:00100000:0.0:1787785065.972126:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785065.972202:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497536 00000100:00100000:0.0:1787785065.972210:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497536 00000100:00100000:0.0:1787785065.972267:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:0.0:1787785065.972277:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668497536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785065.972329:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668497536:12345-0@lo:400:kworker.0 Request processed in 53us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785065.972336:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00080000:0.0:1787785065.972363:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787785065.972397:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540667480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668497536:0@lo:400:kworker.0 00000100:00100000:0.0:1787785065.975249:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668497280:0@lo:400:kworker.0 00000100:00100000:0.0:1787785065.975320:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785065.975396:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497280 00000100:00100000:0.0:1787785065.975404:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497280 00000100:00100000:0.0:1787785065.975472:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:0.0:1787785065.975482:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668497280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785065.975539:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668497280:12345-0@lo:400:kworker.0 Request processed in 58us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785065.975546:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00080000:0.0:1787785065.975578:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785065.975616:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668497280:0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.983337:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785065.983396:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668497408:0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.983459:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785065.983507:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668496896 00000100:00100000:3.0:1787785065.983515:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668496896 00000100:00100000:3.0:1787785065.983567:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:3.0:1787785065.983576:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668496896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.983647:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668496896:12345-0@lo:400:kworker.0 Request processed in 71us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785065.983654:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:3.0:1787785065.983687:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497024 00000100:00100000:3.0:1787785065.983692:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497024 00000100:00100000:3.0:1787785065.983716:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497408 00000100:00100000:3.0:1787785065.983719:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497408 00000100:00100000:0.0:1787785065.983733:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989403125e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668496896:0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.983743:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:3.0:1787785065.983749:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668497024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.983793:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668497024:12345-0@lo:400:kworker.0 Request processed in 44us (11786us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785065.983799:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:3.0:1787785065.983812:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:3.0:1787785065.983818:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668497408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.983849:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989521759680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668497408:12345-0@lo:400:kworker.0 Request processed in 31us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785065.983854:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00080000:3.0:1787785065.983875:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785065.983894:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668497408:0@lo:400:kworker.0 00000100:00100000:1.0:1787785065.984775:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785065.984802:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540666580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668497664:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787785065.984832:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785065.985013:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894031270c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668497024:0@lo:400:kworker.0 00000100:00100000:0.0:1787785065.985206:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497664 00000100:00100000:0.0:1787785065.985213:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497664 00000100:00100000:0.0:1787785065.985250:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:0.0:1787785065.985257:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eba1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668497664:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785065.985287:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785065.985329:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eba1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668497664:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (497us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785065.985336:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:3.0:1787785065.986014:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497152 00000100:00100000:3.0:1787785065.986022:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497152 00000100:00100000:1.0:1787785065.986264:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540666580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668497664:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785065.989253:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:3.0:1787785065.989266:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668497152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785065.989357:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb83c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668497152:12345-0@lo:400:kworker.0 Request processed in 94us (4585us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785065.989364:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:1.0:1787785065.990124:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668497152:0@lo:400:kworker.0 00000100:00080000:0.0:1787785065.994648:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:3.0:1787785066.193837:0:27216:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785066.193847:0:27216:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785066.194374:0:27216:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785066.194382:0:27216:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785066.417713:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785067.272737:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785067.272784:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785067.272952:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395392 00000100:00100000:2.0:1787785067.272963:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395392 00000100:00100000:2.0:1787785067.273019:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 683 00000100:00100000:2.0:1787785067.273029:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658395392:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785067.273098:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2354:x1874627658395392:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 70us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785067.273105:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 683 00000100:00100000:3.0:1787785067.275856:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785067.275893:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785067.275909:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395648 00000100:00100000:2.0:1787785067.275932:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395904 00000100:00100000:2.0:1787785067.275938:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395904 00000100:00100000:0.0:1787785067.275938:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395648 00000100:00100000:0.0:1787785067.275996:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 642 00000100:00100000:0.0:1787785067.276006:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658395648:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785067.276071:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506337c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658395648:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 66us (3286us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785067.276079:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 642 00000100:00100000:3.0:1787785067.276534:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785067.276556:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785067.276580:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785067.276600:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395520 00000100:00100000:3.0:1787785067.276604:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395520 00000100:00100000:0.0:1787785067.276765:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396160 00000100:00100000:0.0:1787785067.276770:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396160 00000100:00100000:0.0:1787785067.278817:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658395776 00000100:00100000:2.0:1787785067.278823:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 658 00000100:00100000:0.0:1787785067.278827:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658395776 00000100:00100000:0.0:1787785067.278853:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396032 00000100:00100000:0.0:1787785067.278856:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396032 00000100:00100000:0.0:1787785067.278876:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:0.0:1787785067.278885:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658396160:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785067.278960:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506334780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658396160:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (3067us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785067.278968:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 643 00000100:00100000:0.0:1787785067.278992:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 644 00000100:00100000:0.0:1787785067.278998:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658395776:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785067.279022:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894030492c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658395776:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (2466us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785067.279028:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 644 00000100:00100000:0.0:1787785067.279039:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 645 00000100:00100000:0.0:1787785067.279045:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658396032:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785067.279067:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530514b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658396032:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (2487us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785067.279073:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 645 00000100:00100000:2.0:1787785067.279158:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658395904:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785067.279210:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506335a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658395904:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 58us (3363us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785067.279217:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 658 00000100:00080000:1.0:1787785067.279440:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785067.279446:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787785067.279448:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785067.279915:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 659 00000100:00100000:3.0:1787785067.279925:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658395520:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785067.280016:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989506336940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658395520:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 92us (3484us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785067.280024:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 659 00010000:00010000:1.0:1787785067.441678:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785068.240846:0:27268:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785068.240856:0:27268:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785068.243448:0:27268:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785068.243468:0:27268:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785068.466171:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787785069.042493:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787785069.042527:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668497792:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787785069.042609:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785069.042690:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497792 00000100:00100000:0.0:1787785069.042699:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497792 00000100:00100000:0.0:1787785069.042768:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1787785069.042779:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668497792:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787785069.042817:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785069.042876:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668497792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 98us (266us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785069.042883:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:3.0:1787785069.043043:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668497792:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787785069.043066:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787785069.489601:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785070.202521:0:27319:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785070.202529:0:27319:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785070.203081:0:27319:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785070.203088:0:27319:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1787785070.513369:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787785071.089729:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eb9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668498176:0@lo:400:kworker.0 00000100:00100000:0.0:1787785071.089800:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785071.089886:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98950895d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668497920:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785071.090077:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785071.090134:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eb9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668498432:0@lo:400:kworker.0 00000100:00100000:3.0:1787785071.090185:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785071.090218:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1787785071.090224:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eba940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668498048:0@lo:400:kworker.0 00000100:00100000:3.0:1787785071.090253:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785071.090273:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eb9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668498560:0@lo:400:kworker.0 00000100:00100000:0.0:1787785071.090273:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536ebad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668498688:0@lo:400:kworker.0 00000100:00100000:3.0:1787785071.090294:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785071.090294:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785071.090319:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787785071.090329:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eb8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668498304:0@lo:400:kworker.0 00000100:00100000:0.0:1787785071.090362:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498432 00000100:00100000:2.0:1787785071.090364:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668497920 00000100:00100000:2.0:1787785071.090368:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668497920 00000100:00100000:0.0:1787785071.090368:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498432 00000100:00100000:0.0:1787785071.090420:0:6669:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498688 00000100:00100000:0.0:1787785071.090423:0:6669:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498688 00000100:00100000:0.0:1787785071.090442:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1787785071.090450:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668498432:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785071.090496:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668498432:12345-0@lo:400:kworker.0 Request processed in 47us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785071.090503:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1787785071.090524:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:0.0:1787785071.090530:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668498688:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785071.090562:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3276:x1874627668498688:12345-0@lo:400:kworker.0 Request processed in 32us (268us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785071.090568:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:2.0:1787785071.095141:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:2.0:1787785071.095174:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668497920:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785071.095213:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785071.095303:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668497920:12345-0@lo:13:osp-pre-1-0.0 Request processed in 137us (5224us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785071.095321:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:2.0:1787785071.095375:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498048 00000100:00100000:2.0:1787785071.095382:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498048 00000100:00100000:2.0:1787785071.095410:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:2.0:1787785071.095416:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668498048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785071.095451:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668498048:12345-0@lo:400:kworker.0 Request processed in 36us (5198us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785071.095457:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00080000:0.0:1787785071.095713:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787785071.095719:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785071.096032:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98950895d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668497920:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785071.096049:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eb9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668498432:0@lo:400:kworker.0 00000100:00100000:3.0:1787785071.096074:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eba940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668498048:0@lo:400:kworker.0 00000100:00100000:0.0:1787785071.097942:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785071.098001:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498176 00000100:00100000:1.0:1787785071.098011:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498176 00000100:00100000:1.0:1787785071.098090:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498560 00000100:00100000:1.0:1787785071.098095:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498560 00000100:00100000:1.0:1787785071.098139:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:3.0:1787785071.098162:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:1.0:1787785071.098162:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668498176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785071.098170:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668498560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785071.098525:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668498176:12345-0@lo:400:kworker.0 Request processed in 377us (8723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785071.098534:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00080000:1.0:1787785071.098561:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785071.099342:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eb9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668498176:0@lo:400:kworker.0 00000100:00100000:0.0:1787785071.099356:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536ebad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668498688:0@lo:400:kworker.0 00000100:00100000:3.0:1787785071.102022:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668498560:12345-0@lo:400:kworker.0 Request processed in 3624us (11498us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785071.102038:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00100000:3.0:1787785071.102072:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498304 00000100:00100000:3.0:1787785071.102080:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498304 00000100:00080000:1.0:1787785071.102218:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785071.102219:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eb9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668498560:0@lo:400:kworker.0 00000100:00100000:2.0:1787785071.102908:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:2.0:1787785071.102923:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668498304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785071.103067:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3276:x1874627668498304:12345-0@lo:400:kworker.0 Request processed in 149us (5130us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785071.103074:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00100000:0.0:1787785071.110140:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eb8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3276:1874627668498304:0@lo:400:kworker.0 00010000:00010000:3.0:1787785071.540410:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785072.133892:0:27370:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785072.133902:0:27370:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785072.135750:0:27370:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785072.135766:0:27370:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785072.393260:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785072.393388:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396288 00000100:00100000:1.0:1787785072.393398:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396288 00000100:00100000:1.0:1787785072.393464:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 685 00000100:00100000:1.0:1787785072.393474:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658396288:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785072.393552:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658396288:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 79us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785072.393561:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 685 00000100:00100000:0.0:1787785072.404621:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785072.404663:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785072.404699:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785072.404726:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785072.404747:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785072.404770:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785072.404796:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396544 00000100:00100000:0.0:1787785072.404805:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396544 00000100:00100000:0.0:1787785072.404855:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397056 00000100:00100000:0.0:1787785072.404858:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397056 00000100:00100000:0.0:1787785072.404874:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396928 00000100:00100000:0.0:1787785072.404876:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396928 00000100:00100000:0.0:1787785072.404887:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396672 00000100:00100000:0.0:1787785072.404889:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396672 00000100:00100000:0.0:1787785072.404922:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 648 00000100:00100000:0.0:1787785072.404932:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658396544:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785072.404999:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98950895d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658396544:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 68us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785072.405007:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 648 00000100:00100000:0.0:1787785072.405028:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00100000:0.0:1787785072.405034:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658397056:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785072.405058:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894031270c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658397056:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785072.405064:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 649 00000100:00100000:0.0:1787785072.405075:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 650 00000100:00100000:0.0:1787785072.405080:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658396928:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785072.405105:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658396928:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785072.405111:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 650 00000100:00100000:0.0:1787785072.405121:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 651 00000100:00100000:0.0:1787785072.405127:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658396672:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785072.405180:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658396672:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (376us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785072.405187:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 651 00000100:00100000:3.0:1787785072.405462:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396416 00000100:00100000:3.0:1787785072.405466:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396416 00000100:00100000:3.0:1787785072.405498:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658396800 00000100:00100000:3.0:1787785072.405501:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658396800 00000100:00100000:3.0:1787785072.405522:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:3.0:1787785072.405529:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989508340c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658396416:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785072.405559:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989508340c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2354:x1874627658396416:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 32us (948us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785072.405588:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:2.0:1787785072.406943:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 664 00000100:00100000:2.0:1787785072.406962:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658396800:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785072.407014:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989403125a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658396800:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (2315us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785072.407023:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 664 00000100:00080000:1.0:1787785072.407985:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787785072.407993:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785072.407995:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787785072.561474:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787785073.586632:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785073.909409:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785073.909418:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785073.910433:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785073.910447:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1787785074.033413:0:27422:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785074.033423:0:27422:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785074.033963:0:27422:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785074.033972:0:27422:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785074.162392:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668498816:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787785074.162458:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785074.162646:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498816 00000100:00100000:2.0:1787785074.162654:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498816 00000100:00100000:2.0:1787785074.162720:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:2.0:1787785074.162729:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668498816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785074.162764:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785074.162809:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668498816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785074.162815:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00080000:0.0:1787785074.162894:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785074.162930:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668498816:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787785074.609589:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787785075.639279:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785075.964241:0:27473:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785075.964251:0:27473:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785075.965649:0:27473:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785075.965673:0:27473:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785076.209351:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eb8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668498944:0@lo:400:kworker.0 00000100:00100000:2.0:1787785076.209358:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668499072:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785076.209427:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785076.209490:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787785076.209498:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668499456:0@lo:400:kworker.0 00000100:00100000:3.0:1787785076.209523:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785076.209539:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785076.209588:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499072 00000100:00100000:2.0:1787785076.209594:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499072 00000100:00100000:2.0:1787785076.209638:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:2.0:1787785076.209645:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668499072:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787785076.209676:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785076.209714:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668499072:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785076.209721:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:2.0:1787785076.209765:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668499072:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785076.209853:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668498944 00000100:00100000:2.0:1787785076.209856:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668498944 00000100:00100000:2.0:1787785076.209878:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:2.0:1787785076.209884:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668498944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.212995:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499456 00000100:00100000:0.0:1787785076.213007:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499456 00000100:00100000:0.0:1787785076.213065:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:0.0:1787785076.213074:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668499456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.213168:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668499456:12345-0@lo:400:kworker.0 Request processed in 96us (3630us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785076.213176:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00080000:0.0:1787785076.213199:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787785076.213203:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787785076.213242:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668499456:0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.213287:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eb83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668499328:0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.213321:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785076.213345:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787785076.213351:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536ebb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668499200:0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.213373:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785076.214026:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499328 00000100:00100000:0.0:1787785076.214033:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499328 00000100:00100000:0.0:1787785076.214072:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499200 00000100:00100000:0.0:1787785076.214074:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499200 00000100:00100000:0.0:1787785076.214095:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:0.0:1787785076.214101:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668499328:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.214144:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540667480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668499328:12345-0@lo:400:kworker.0 Request processed in 43us (822us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785076.214177:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:0.0:1787785076.214192:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:0.0:1787785076.214197:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668499200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.214220:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668499200:12345-0@lo:400:kworker.0 Request processed in 24us (847us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785076.214226:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00080000:0.0:1787785076.214243:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785076.214267:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536ebb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668499200:0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.214276:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eb83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668499328:0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.214528:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eb8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668499584:0@lo:400:kworker.0 00000100:00100000:0.0:1787785076.214569:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785076.214701:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499584 00000100:00100000:1.0:1787785076.214705:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499584 00000100:00100000:1.0:1787785076.214746:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 667 00000100:00100000:1.0:1787785076.214752:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668499584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785076.214790:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540666580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668499584:12345-0@lo:400:kworker.0 Request processed in 38us (221us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785076.214796:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 667 00000100:00100000:1.0:1787785076.214886:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eb8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668499584:0@lo:400:kworker.0 00000100:00080000:0.0:1787785076.214940:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785076.220228:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eb8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668498944:0@lo:400:kworker.0 00000100:00100000:2.0:1787785076.220393:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668498944:12345-0@lo:400:kworker.0 Request processed in 10506us (10868us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785076.220466:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00010000:00010000:3.0:1787785076.657822:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787785077.517777:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785077.517843:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785077.517877:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785077.517899:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785077.517928:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785077.517949:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785077.517977:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785077.518033:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397440 00000100:00100000:3.0:1787785077.518041:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397184 00000100:00100000:3.0:1787785077.518049:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397184 00000100:00100000:1.0:1787785077.518060:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397440 00000100:00100000:1.0:1787785077.518117:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397952 00000100:00100000:1.0:1787785077.518121:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397952 00000100:00100000:3.0:1787785077.518130:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 687 00000100:00100000:3.0:1787785077.518179:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658397184:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785077.518187:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397568 00000100:00100000:1.0:1787785077.518191:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397568 00000100:00100000:1.0:1787785077.518210:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397824 00000100:00100000:1.0:1787785077.518213:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397824 00000100:00100000:1.0:1787785077.518233:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 653 00000100:00100000:1.0:1787785077.518254:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658397440:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785077.518277:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658397184:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 135us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785077.518287:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 687 00000100:00100000:1.0:1787785077.518301:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb8f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658397440:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (456us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785077.518310:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 653 00000100:00100000:1.0:1787785077.518330:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 654 00000100:00100000:1.0:1787785077.518338:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536ebad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658397952:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785077.518366:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536ebad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658397952:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 29us (466us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785077.518373:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 654 00000100:00100000:1.0:1787785077.518388:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 655 00000100:00100000:1.0:1787785077.518397:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eba940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658397568:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785077.518425:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eba940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658397568:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 30us (474us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785077.518431:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 655 00000100:00100000:1.0:1787785077.518444:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 656 00000100:00100000:1.0:1787785077.518451:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658397824:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785077.518477:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658397824:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785077.518498:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 656 00000100:00100000:1.0:1787785077.518520:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397696 00000100:00100000:1.0:1787785077.518524:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397696 00000100:00080000:3.0:1787785077.518555:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787785077.518562:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787785077.518565:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787785077.518570:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658397312 00000100:00100000:1.0:1787785077.518575:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658397312 00000100:00100000:3.0:1787785077.518595:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 668 00000100:00100000:3.0:1787785077.518606:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658397696:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785077.518619:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00100000:1.0:1787785077.518629:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658397312:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785077.518664:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658397696:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 60us (785us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785077.518670:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658397312:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 40us (738us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785077.518678:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 668 00000100:00100000:1.0:1787785077.518685:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00080000:1.0:1787785077.518725:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787785077.681357:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785077.965884:0:27524:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785077.965905:0:27524:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785077.966462:0:27524:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785077.966472:0:27524:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785078.706046:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785079.281555:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668499712:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787785079.281666:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785079.282103:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499712 00000100:00100000:0.0:1787785079.282115:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499712 00000100:00100000:0.0:1787785079.282225:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:0.0:1787785079.282237:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668499712:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787785079.282284:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785079.282348:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668499712:12345-0@lo:13:osp-pre-0-0.0 Request processed in 114us (681us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785079.282356:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:0.0:1787785079.282424:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668499712:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1787785079.282978:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787785079.730666:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785080.133078:0:27575:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785080.133093:0:27575:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785080.133668:0:27575:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785080.133693:0:27575:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785080.755293:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785081.329625:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540665a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668499840:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785081.329732:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785081.329892:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499840 00000100:00100000:0.0:1787785081.329903:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499840 00000100:00100000:0.0:1787785081.329993:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:0.0:1787785081.330006:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668499840:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785081.330055:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785081.330121:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668499840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785081.330130:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:0.0:1787785081.330335:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540665a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668499840:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1787785081.330557:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787785081.331463:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668499968:0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.331591:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785081.331641:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668500480:0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.331673:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785081.331702:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787785081.331710:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668500096:0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.331741:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785081.331741:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668500352:0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.331783:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668499968 00000100:00100000:0.0:1787785081.331800:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785081.331803:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668499968 00000100:00100000:0.0:1787785081.331832:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787785081.331840:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668500224:0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.331862:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 688 00000100:00100000:0.0:1787785081.331870:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785081.331876:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668499968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785081.331906:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500352 00000100:00100000:0.0:1787785081.331911:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500352 00000100:00100000:3.0:1787785081.331934:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668499968:12345-0@lo:400:kworker.0 Request processed in 61us (342us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785081.331943:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 688 00000100:00100000:0.0:1787785081.331966:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:0.0:1787785081.331976:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668500352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.331989:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500480 00000100:00100000:3.0:1787785081.331995:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500480 00000100:00100000:0.0:1787785081.332025:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540665a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668500352:12345-0@lo:400:kworker.0 Request processed in 51us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785081.332032:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500096 00000100:00100000:0.0:1787785081.332035:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:3.0:1787785081.332038:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500096 00000100:00100000:3.0:1787785081.332063:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500224 00000100:00100000:3.0:1787785081.332066:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500224 00000100:00100000:0.0:1787785081.332074:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:0.0:1787785081.332087:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668500480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.332090:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:3.0:1787785081.332102:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668500096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785081.332128:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3278:x1874627668500480:12345-0@lo:400:kworker.0 Request processed in 45us (453us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785081.332137:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:3.0:1787785081.332169:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668500096:12345-0@lo:400:kworker.0 Request processed in 68us (428us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785081.332177:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00000100:00100000:0.0:1787785081.333280:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:0.0:1787785081.333318:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668500224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785081.333395:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406661c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668500224:12345-0@lo:400:kworker.0 Request processed in 97us (1525us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785081.335447:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668500096:0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.335471:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668499968:0@lo:400:kworker.0 00000100:00100000:3.0:1787785081.335489:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3278:1874627668500480:0@lo:400:kworker.0 00000100:00100000:0.0:1787785081.339579:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00080000:0.0:1787785081.339641:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787785081.339650:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785081.339726:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668500224:0@lo:400:kworker.0 00000100:00100000:0.0:1787785081.339745:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668500352:0@lo:400:kworker.0 00010000:00010000:3.0:1787785081.777388:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785082.081458:0:27627:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785082.081474:0:27627:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785082.082018:0:27627:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785082.082039:0:27627:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787785082.634283:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785082.634334:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785082.634375:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785082.634422:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785082.634456:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785082.634493:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785082.634590:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785082.634630:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398080 00000100:00100000:1.0:1787785082.634642:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398080 00000100:00100000:1.0:1787785082.634709:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 689 00000100:00100000:1.0:1787785082.634720:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658398080:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785082.634806:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658398080:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 88us (384us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785082.634815:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 689 00000100:00100000:1.0:1787785082.634858:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398464 00000100:00100000:1.0:1787785082.634862:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398464 00000100:00100000:1.0:1787785082.634899:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398720 00000100:00100000:1.0:1787785082.634902:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398720 00000100:00100000:1.0:1787785082.634921:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398336 00000100:00100000:1.0:1787785082.634924:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398336 00000100:00100000:1.0:1787785082.634939:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398848 00000100:00100000:1.0:1787785082.634953:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398848 00000100:00100000:1.0:1787785082.634971:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 658 00000100:00100000:1.0:1787785082.634978:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658398464:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785082.635180:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 659 00000100:00100000:0.0:1787785082.635193:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536ebb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658398720:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785082.635230:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536ebb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2355:x1874627658398720:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 42us (855us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785082.635238:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 659 00000100:00100000:0.0:1787785082.635253:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 660 00000100:00100000:0.0:1787785082.635260:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536ebb840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658398336:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785082.635287:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536ebb840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2355:x1874627658398336:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (829us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785082.635293:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 660 00000100:00100000:0.0:1787785082.635304:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 661 00000100:00100000:0.0:1787785082.635310:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb83c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658398848:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785082.635333:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb83c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2355:x1874627658398848:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (743us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785082.635340:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 661 00000100:00080000:0.0:1787785082.635357:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787785082.635362:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787785082.636083:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2355:x1874627658398464:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 1098us (1742us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785082.636094:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 658 00000100:00100000:3.0:1787785082.636759:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398208 00000100:00100000:3.0:1787785082.636770:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398208 00000100:00100000:3.0:1787785082.636835:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398592 00000100:00100000:3.0:1787785082.636839:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398592 00000100:00100000:3.0:1787785082.637637:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00100000:3.0:1787785082.637655:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536ebbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658398208:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785082.637732:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536ebbc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2355:x1874627658398208:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 84us (3465us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785082.637739:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00100000:3.0:1787785082.637758:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 674 00000100:00100000:3.0:1787785082.637765:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536eb8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658398592:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785082.637790:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536eb8780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2355:x1874627658398592:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 26us (3297us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785082.637797:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 674 00000100:00080000:1.0:1787785082.638047:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787785082.801327:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785083.122304:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785083.122313:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785083.122791:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785083.122805:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787785083.830732:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785083.964282:0:27678:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785083.964292:0:27678:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785083.965545:0:27678:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785083.965563:0:27678:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787785084.404044:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668500608:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787785084.404129:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785084.404295:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500608 00000100:00100000:0.0:1787785084.404303:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500608 00000100:00100000:0.0:1787785084.404368:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:0.0:1787785084.404380:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989536ebb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668500608:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787785084.404412:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00080000:1.0:1787785084.404421:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785084.404458:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989536ebb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668500608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785084.404479:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:1.0:1787785084.404613:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668500608:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787785084.849431:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785085.863361:0:27729:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785085.863374:0:27729:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785085.863976:0:27729:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785085.863990:0:27729:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785085.875878:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787785086.449452:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787785086.449783:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668500864:0@lo:400:kworker.0 00000100:00100000:2.0:1787785086.449884:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785086.449951:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668500736:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785086.449982:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785086.450062:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500864 00000100:00100000:3.0:1787785086.450072:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500864 00000100:00100000:3.0:1787785086.450144:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 690 00000100:00100000:3.0:1787785086.450172:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668500864:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785086.450236:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895345be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668500864:12345-0@lo:400:kworker.0 Request processed in 66us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785086.450244:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 690 00000100:00100000:0.0:1787785086.450285:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500736 00000100:00100000:0.0:1787785086.450341:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500736 00000100:00100000:2.0:1787785086.450426:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668500864:0@lo:400:kworker.0 00000100:00100000:0.0:1787785086.450460:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:0.0:1787785086.450471:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668500736:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785086.450523:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785086.450578:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668500736:12345-0@lo:13:osp-pre-1-0.0 Request processed in 110us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785086.450652:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00080000:3.0:1787785086.450691:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787785086.452021:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668500736:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787785086.461799:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668500992:0@lo:400:kworker.0 00000100:00100000:3.0:1787785086.461804:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539374b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668501248:0@lo:400:kworker.0 00000100:00100000:1.0:1787785086.461815:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895393752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668501120:0@lo:400:kworker.0 00000100:00100000:2.0:1787785086.461897:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539376940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668501376:0@lo:400:kworker.0 00000100:00100000:3.0:1787785086.465547:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785086.465671:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668501248 00000100:00100000:3.0:1787785086.465682:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668501248 00000100:00100000:3.0:1787785086.465767:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:3.0:1787785086.465779:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668501248:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785086.465856:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668501248:12345-0@lo:400:kworker.0 Request processed in 80us (324us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785086.465864:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00080000:3.0:1787785086.465896:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785086.465936:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539374b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668501248:0@lo:400:kworker.0 00000100:00100000:1.0:1787785086.466197:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785086.466343:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668501120 00000100:00100000:3.0:1787785086.466348:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668501120 00000100:00100000:3.0:1787785086.466397:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:3.0:1787785086.466405:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668501120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785086.466596:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3279:x1874627668501120:12345-0@lo:400:kworker.0 Request processed in 190us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785086.466604:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00000100:00100000:3.0:1787785086.466641:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895393752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3279:1874627668501120:0@lo:400:kworker.0 00000100:00080000:1.0:1787785086.467263:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787785086.471896:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785086.471975:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785086.472038:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668501376 00000100:00100000:0.0:1787785086.472050:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668501376 00000100:00100000:3.0:1787785086.472267:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668500992 00000100:00100000:3.0:1787785086.472274:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668500992 00000100:00100000:3.0:1787785086.472408:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:3.0:1787785086.472419:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668500992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785086.472508:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406652c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3278:x1874627668500992:12345-0@lo:400:kworker.0 Request processed in 90us (533us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785086.472516:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00080000:3.0:1787785086.472887:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785086.473014:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:0.0:1787785086.473025:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668501376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785086.473092:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668501376:12345-0@lo:400:kworker.0 Request processed in 70us (1207us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785086.473100:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:0.0:1787785086.473169:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3278:1874627668500992:0@lo:400:kworker.0 00000100:00100000:3.0:1787785086.473179:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539376940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668501376:0@lo:400:kworker.0 00010000:00010000:2.0:1787785086.897488:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785087.756522:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785087.756563:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785087.756589:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785087.756612:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785087.756628:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785087.756647:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785087.756726:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785087.756761:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658398976 00000100:00100000:3.0:1787785087.756773:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658398976 00000100:00100000:1.0:1787785087.757201:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658399360 00000100:00100000:1.0:1787785087.757207:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658399360 00000100:00100000:1.0:1787785087.757263:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658399616 00000100:00100000:1.0:1787785087.757266:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658399616 00000100:00100000:1.0:1787785087.757285:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658399232 00000100:00100000:1.0:1787785087.757288:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658399232 00000100:00100000:1.0:1787785087.757301:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658399744 00000100:00100000:1.0:1787785087.757304:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658399744 00000100:00100000:1.0:1787785087.757319:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:1.0:1787785087.757331:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658399360:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785087.757423:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d83c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658399360:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 93us (832us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785087.757432:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 663 00000100:00100000:1.0:1787785087.757453:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 664 00000100:00100000:1.0:1787785087.757459:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658399616:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785087.757497:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658399616:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 37us (884us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785087.757504:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 664 00000100:00100000:1.0:1787785087.757517:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 665 00000100:00100000:1.0:1787785087.757523:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658399232:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785087.757544:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658399232:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (916us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785087.757550:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 665 00000100:00100000:1.0:1787785087.757559:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 666 00000100:00100000:1.0:1787785087.757564:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658399744:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785087.757582:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658399744:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 18us (856us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785087.757588:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 666 00000100:00100000:1.0:1787785087.757606:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658399104 00000100:00100000:1.0:1787785087.757609:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658399104 00000100:00100000:1.0:1787785087.757637:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658399488 00000100:00100000:1.0:1787785087.757640:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658399488 00000100:00100000:1.0:1787785087.757660:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 678 00000100:00100000:1.0:1787785087.757667:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658399104:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785087.757694:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658399104:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (1131us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785087.757700:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 678 00000100:00100000:1.0:1787785087.757711:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 679 00000100:00100000:1.0:1787785087.757717:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658399488:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785087.757737:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035da940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658399488:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (1089us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785087.757743:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 679 00000100:00100000:3.0:1787785087.758478:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 691 00000100:00100000:3.0:1787785087.758491:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658398976:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785087.758640:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658398976:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 153us (2132us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785087.758649:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 691 00000100:00080000:3.0:1787785087.758695:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787785087.758701:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787785087.758703:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000040:00080000:3.0:1787785087.916275:0:27780:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785087.916288:0:27780:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785087.923178:0:27780:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785087.923203:0:27780:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785087.925944:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787785088.948093:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785089.523488:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787785089.523517:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668501504:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787785089.523606:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785089.525494:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668501504 00000100:00100000:2.0:1787785089.525513:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668501504 00000100:00100000:2.0:1787785089.525616:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:2.0:1787785089.525629:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668501504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785089.525681:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785089.525783:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035dbc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668501504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 157us (2176us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785089.525790:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00080000:2.0:1787785089.525815:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785089.525873:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668501504:0@lo:13:osp-pre-0-0.0 00000040:00080000:0.0:1787785089.897364:0:27832:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785089.897376:0:27832:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785089.897903:0:27832:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785089.897917:0:27832:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1787785089.969774:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787785090.996818:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785091.569566:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787785091.569605:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668501632:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785091.569713:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785091.569876:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668501632 00000100:00100000:0.0:1787785091.569887:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668501632 00000100:00100000:0.0:1787785091.569980:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:0.0:1787785091.569993:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668501632:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785091.570045:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785091.570107:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035db0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668501632:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785091.570116:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:0.0:1787785091.570210:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668501632:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1787785091.570280:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787785091.571005:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668501760:0@lo:400:kworker.0 00000100:00100000:3.0:1787785091.571065:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785091.571830:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668501760 00000100:00100000:2.0:1787785091.571841:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668501760 00000100:00100000:2.0:1787785091.571906:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 692 00000100:00100000:2.0:1787785091.571917:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668501760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785091.574837:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668501888:0@lo:400:kworker.0 00000100:00100000:1.0:1787785091.575420:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668502016:0@lo:400:kworker.0 00000100:00100000:1.0:1787785091.575548:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785091.575608:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787785091.575617:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668502144:0@lo:400:kworker.0 00000100:00100000:1.0:1787785091.575649:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785091.575702:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502016 00000100:00100000:1.0:1787785091.575712:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502016 00000100:00100000:1.0:1787785091.575791:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 680 00000100:00100000:1.0:1787785091.575803:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668502016:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785091.575884:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539377480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668502016:12345-0@lo:400:kworker.0 Request processed in 83us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785091.575891:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 680 00000100:00100000:1.0:1787785091.575922:0:6670:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502144 00000100:00100000:1.0:1787785091.575925:0:6670:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502144 00000100:00100000:1.0:1787785091.575958:0:6670:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 667 00000100:00100000:1.0:1787785091.575964:0:6670:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393761c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668502144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785091.575996:0:6670:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393761c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668502144:12345-0@lo:400:kworker.0 Request processed in 32us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785091.576003:0:6670:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 667 00000100:00100000:1.0:1787785091.576058:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668502144:0@lo:400:kworker.0 00000100:00100000:1.0:1787785091.576070:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668502016:0@lo:400:kworker.0 00000100:00100000:0.0:1787785091.576669:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785091.576704:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787785091.576716:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668502272:0@lo:400:kworker.0 00000100:00100000:0.0:1787785091.576745:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785091.576794:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668501888 00000100:00100000:0.0:1787785091.576800:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668501888 00000100:00100000:0.0:1787785091.576837:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502272 00000100:00100000:0.0:1787785091.576840:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502272 00000100:00100000:0.0:1787785091.576865:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 681 00000100:00100000:0.0:1787785091.576872:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668501888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785091.576917:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021af0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668501888:12345-0@lo:400:kworker.0 Request processed in 44us (248us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785091.576924:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 681 00000100:00100000:0.0:1787785091.576947:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:0.0:1787785091.576953:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668502272:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785091.576978:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668502272:12345-0@lo:400:kworker.0 Request processed in 26us (232us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785091.576984:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:0.0:1787785091.577020:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668502272:0@lo:400:kworker.0 00000100:00100000:0.0:1787785091.577030:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989536eba580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668501888:0@lo:400:kworker.0 00000100:00080000:0.0:1787785091.577422:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787785091.577430:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787785091.578966:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668501760:12345-0@lo:400:kworker.0 Request processed in 6471us (7321us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785091.578986:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 692 00000100:00100000:3.0:1787785091.579144:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989540664f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668501760:0@lo:400:kworker.0 00000100:00100000:3.0:1787785091.656564:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785091.664766:0:11303:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658399872 00000100:00100000:1.0:1787785091.664790:0:11303:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658399872 00000100:00100000:2.0:1787785091.665023:0:11303:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 78 00000100:00100000:2.0:1787785091.665036:0:11303:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+111:2357:x1874627658399872:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787785091.665057:0:11303:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 16 locks, starting at 0 00000100:00080000:0.0:1787785091.665316:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787785091.665487:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904200/0xb0ce63137acc2e44 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b009a expref: 111 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785091.665549:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x10:0x0].0x0 (ffff989510520a00) 00010000:00010000:2.0:1787785091.665569:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2200/0xb0ce63137acc2bf8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xf:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afff2 expref: 110 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785091.665656:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff98950a38ca00) 00010000:00010000:3.0:1787785091.665792:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2c00/0xb0ce63137acc2b6c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841affcf expref: 109 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.665823:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114800/0xb0ce63137acc2a5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x51/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841aff82 expref: 108 pid: 5813 timeout: 421 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.665836:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff989519d44900) 00010000:00010000:3.0:1787785091.665847:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654600/0xb0ce63137acc2920 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841aff20 expref: 107 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.665856:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc:0x0].0x0 (ffff989519d44e00) 00010000:00010000:3.0:1787785091.665963:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2800/0xb0ce63137acc288d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afefd expref: 106 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.665986:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905400/0xb0ce63137acc274b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afebe expref: 105 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.665996:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff989531111700) 00010000:00010000:3.0:1787785091.666004:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b2600/0xb0ce63137acc271a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afeb0 expref: 104 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666018:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff98950a38ce00) 00010000:00010000:3.0:1787785091.666028:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a115200/0xb0ce63137acc26b1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afe9b expref: 103 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666036:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff989510520f00) 00010000:00010000:3.0:1787785091.666046:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953efb3800/0xb0ce63137acc2473 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afdf3 expref: 102 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666054:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff989510520500) 00010000:00010000:3.0:1787785091.666060:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654e00/0xb0ce63137acc230e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afd8a expref: 101 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666070:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff989519d44800) 00010000:00010000:3.0:1787785091.666085:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135400/0xb0ce63137acc2219 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afd36 expref: 100 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666094:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff98950a38cb00) 00010000:00010000:3.0:1787785091.666101:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135800/0xb0ce63137acc20e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afce2 expref: 99 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666112:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953efb3e00/0xb0ce63137acc204b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afcc6 expref: 98 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666124:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98950166f400/0xb0ce63137acc1f5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afc95 expref: 97 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666137:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4:0x0].0x0 (ffff989519d44a00) 00010000:00010000:3.0:1787785091.666186:0:11303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98952a114400/0xb0ce63137acc1f2c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afc87 expref: 96 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787785091.666197:0:11303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff989510520300) 00010000:00010000:3.0:1787785091.666203:0:11303:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787785091.666532:0:11303:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2357:x1874627658399872:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 1496us (9987us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785091.666543:0:11303:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 78 00000040:00080000:3.0:1787785091.885549:0:27883:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785091.885559:0:27883:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785091.886088:0:27883:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785091.886096:0:27883:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785092.018045:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785092.873823:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785092.873866:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785092.873905:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785092.873944:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785092.873956:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400000 00000100:00100000:1.0:1787785092.873975:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400000 00000100:00100000:3.0:1787785092.873976:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785092.874007:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785092.874064:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 693 00000100:00100000:1.0:1787785092.874078:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658400000:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785092.874104:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785092.874141:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400384 00000100:00100000:3.0:1787785092.874193:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400384 00000100:00100000:1.0:1787785092.874220:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2355:x1874627658400000:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 143us (404us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785092.874232:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 693 00000100:00100000:3.0:1787785092.874257:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400640 00000100:00100000:3.0:1787785092.874262:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400640 00000100:00100000:2.0:1787785092.874329:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400128 00000100:00100000:2.0:1787785092.874337:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400128 00000100:00100000:2.0:1787785092.874373:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400512 00000100:00100000:2.0:1787785092.874377:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400512 00000100:00100000:2.0:1787785092.874403:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 683 00000100:00100000:2.0:1787785092.874413:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2355:x1874627658400128:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785092.874487:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895406643c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2355:x1874627658400128:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 75us (619us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785092.874495:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 683 00000100:00100000:2.0:1787785092.874513:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 684 00000100:00100000:2.0:1787785092.874519:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658400512:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785092.874543:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2354:x1874627658400512:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 25us (538us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785092.874550:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 684 00000100:00100000:3.0:1787785092.874579:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400256 00000100:00100000:3.0:1787785092.874584:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400256 00000100:00100000:1.0:1787785092.876093:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400768 00000100:00100000:1.0:1787785092.876110:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400768 00000100:00100000:1.0:1787785092.876195:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 668 00000100:00100000:1.0:1787785092.876208:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658400384:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785092.876568:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989540664f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2354:x1874627658400384:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 361us (2661us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785092.876578:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 668 00000100:00100000:1.0:1787785092.876602:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00100000:1.0:1787785092.876609:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658400640:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785092.876738:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391303c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2354:x1874627658400640:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 129us (2798us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785092.876746:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 669 00000100:00100000:1.0:1787785092.876758:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 670 00000100:00100000:1.0:1787785092.876765:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658400256:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785092.876784:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2354:x1874627658400256:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (2810us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785092.876790:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 670 00000100:00100000:1.0:1787785092.876800:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 671 00000100:00100000:1.0:1787785092.876805:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658400768:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785092.876874:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2354:x1874627658400768:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 68us (2773us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785092.876880:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 671 00000100:00080000:1.0:1787785092.876941:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787785092.876948:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787785092.876950:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787785093.046533:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785093.364103:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785093.364114:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785093.364970:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785093.364989:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1787785093.741075:0:27934:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785093.741084:0:27934:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785093.741920:0:27934:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785093.741933:0:27934:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1787785094.065898:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785094.641488:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787785094.641511:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668502400:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787785094.641577:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785094.641707:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502400 00000100:00100000:2.0:1787785094.641716:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502400 00000100:00100000:2.0:1787785094.641783:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:2.0:1787785094.641793:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668502400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785094.641830:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785094.641881:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668502400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785094.641889:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:2.0:1787785094.641934:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668502400:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787785094.642184:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787785095.093902:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785095.700276:0:27986:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785095.700285:0:27986:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785095.703507:0:27986:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785095.703527:0:27986:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785096.122782:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787785096.690654:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668502528:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787785096.690727:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785096.690795:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787785096.690800:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668502656:0@lo:400:kworker.0 00000100:00100000:2.0:1787785096.690801:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668502784:0@lo:400:kworker.0 00000100:00100000:2.0:1787785096.690837:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785096.690844:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785096.690875:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502656 00000100:00100000:1.0:1787785096.690875:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502784 00000100:00100000:2.0:1787785096.690882:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502656 00000100:00100000:1.0:1787785096.690882:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502784 00000100:00100000:2.0:1787785096.690922:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:1.0:1787785096.690922:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:2.0:1787785096.690930:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668502656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785096.690930:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668502784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787785096.691013:0:6673:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502528 00000100:00100000:0.0:1787785096.691018:0:6673:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502528 00000100:00100000:0.0:1787785096.691056:0:6673:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:0.0:1787785096.691063:0:6673:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668502528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785096.691091:0:6673:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1787785096.691214:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787785096.691223:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668503040:0@lo:400:kworker.0 00000100:00100000:1.0:1787785096.701854:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3276:x1874627668502784:12345-0@lo:400:kworker.0 Request processed in 10860us (10951us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785096.701869:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00080000:1.0:1787785096.701926:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787785096.712627:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895391312c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668502912:0@lo:400:kworker.0 00000100:00100000:1.0:1787785096.712711:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785096.712811:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3276:1874627668502784:0@lo:400:kworker.0 00000100:00100000:1.0:1787785096.712851:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668502912 00000100:00100000:1.0:1787785096.712858:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668502912 00000100:00100000:1.0:1787785096.712910:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:1.0:1787785096.712919:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668502912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785096.713187:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785096.713225:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503040 00000100:00100000:3.0:1787785096.713229:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503040 00000100:00100000:3.0:1787785096.713259:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:3.0:1787785096.713267:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668503040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787785096.713303:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668503040:12345-0@lo:400:kworker.0 Request processed in 38us (116us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785096.713310:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00100000:3.0:1787785096.713342:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668503040:0@lo:400:kworker.0 00000100:00100000:3.0:1787785096.713386:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668503168:0@lo:400:kworker.0 00000100:00100000:1.0:1787785096.714612:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539374b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668502912:12345-0@lo:400:kworker.0 Request processed in 1688us (1895us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785096.714623:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00080000:1.0:1787785096.714655:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787785096.714660:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787785096.714688:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895391312c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668502912:0@lo:400:kworker.0 00000100:00100000:3.0:1787785096.714837:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785096.714974:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3278:x1874627668502656:12345-0@lo:400:kworker.0 Request processed in 24042us (24126us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785096.714983:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:1.0:1787785096.716536:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503168 00000100:00100000:1.0:1787785096.716548:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503168 00000100:00100000:1.0:1787785096.716621:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 687 00000100:00100000:1.0:1787785096.716630:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668503168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785096.716703:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668503168:12345-0@lo:400:kworker.0 Request processed in 74us (1866us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785096.716710:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 687 00000100:00100000:3.0:1787785096.716819:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539133c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668503168:0@lo:400:kworker.0 00000100:00100000:3.0:1787785096.716852:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:a5a93c05-6249-4607-a205-909a326a8fd1:3278:1874627668502656:0@lo:400:kworker.0 00000100:00080000:2.0:1787785096.717592:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785096.717700:0:6673:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895393752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668502528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 26635us (26970us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785096.717710:0:6673:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:1.0:1787785096.718208:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668502528:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1787785097.137529:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785097.785386:0:28037:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785097.790944:0:28037:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785097.792460:0:28037:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785097.792478:0:28037:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785097.995924:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785097.996069:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785097.996174:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401024 00000100:00100000:2.0:1787785097.996184:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401024 00000100:00080000:3.0:1787785097.996940:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785097.997010:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 688 00000100:00100000:3.0:1787785097.997023:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658401024:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785097.997100:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658401024:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 80us (1030us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785097.997108:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 688 00000100:00100000:3.0:1787785097.997175:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:3.0:1787785097.997276:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401152 00000100:00100000:3.0:1787785097.997281:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401152 00000100:00100000:3.0:1787785097.997318:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00100000:3.0:1787785097.997325:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658401152:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785097.997366:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391330c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658401152:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 41us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785097.997372:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 673 00000100:00080000:3.0:1787785097.997390:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787785097.999081:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658400896 00000100:00100000:1.0:1787785097.999093:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658400896 00000100:00100000:0.0:1787785097.999289:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 695 00000100:00100000:0.0:1787785097.999298:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658400896:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785097.999369:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658400896:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 72us (3455us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785097.999377:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 695 00000100:00100000:3.0:1787785098.006957:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785098.007589:0:13825:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401280 00000100:00100000:1.0:1787785098.007599:0:13825:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401280 00000100:00100000:1.0:1787785098.007817:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 674 00000100:00100000:1.0:1787785098.007831:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658401280:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785098.007907:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539131680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658401280:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 80us (961us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785098.007914:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 674 00000100:00080000:3.0:1787785098.010785:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787785098.012575:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785098.013179:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401536 00000100:00100000:1.0:1787785098.013190:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401536 00000100:00100000:1.0:1787785098.013264:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 675 00000100:00100000:1.0:1787785098.013276:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658401536:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785098.013349:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539133c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658401536:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 77us (783us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785098.013357:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 675 00000100:00080000:2.0:1787785098.013561:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787785098.013765:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785098.013799:0:3270:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785098.014734:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401408 00000100:00100000:1.0:1787785098.014744:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401408 00000100:00100000:3.0:1787785098.014849:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401664 00000100:00100000:3.0:1787785098.014854:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401664 00000100:00100000:0.0:1787785098.016056:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 689 00000100:00100000:0.0:1787785098.016071:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658401408:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785098.016172:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658401408:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 105us (2407us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785098.016180:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 689 00000100:00080000:2.0:1787785098.016324:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787785098.016330:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787785098.016544:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 676 00000100:00100000:3.0:1787785098.016554:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658401664:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785098.016585:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658401664:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 34us (2786us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785098.016595:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 676 00010000:00010000:0.0:1787785098.163091:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787785099.185545:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785099.522257:0:28088:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785099.522267:0:28088:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785099.522795:0:28088:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785099.522810:0:28088:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785099.761361:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787785099.761396:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668503296:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787785099.761465:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785099.761605:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503296 00000100:00100000:0.0:1787785099.761614:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503296 00000100:00100000:0.0:1787785099.761683:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1787785099.761694:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668503296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787785099.761726:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785099.761776:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895391312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668503296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785099.761784:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00080000:2.0:1787785099.762049:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787785099.762232:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668503296:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787785100.209505:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787785101.233463:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785101.266486:0:28139:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785101.266497:0:28139:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785101.267022:0:28139:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785101.267037:0:28139:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787785101.810872:0:3276:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787785101.810895:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894021ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668503424:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787785101.810962:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785101.811254:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668503936:0@lo:400:kworker.0 00000100:00100000:2.0:1787785101.811288:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785101.811316:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787785101.811322:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668503808:0@lo:400:kworker.0 00000100:00100000:2.0:1787785101.811345:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785101.811447:0:22706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503936 00000100:00100000:2.0:1787785101.811454:0:22706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503936 00000100:00100000:2.0:1787785101.811506:0:22706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:2.0:1787785101.811515:0:22706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668503936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785101.811573:0:22706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668503936:12345-0@lo:400:kworker.0 Request processed in 60us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785101.811580:0:22706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00080000:2.0:1787785101.811600:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785101.811638:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668503936:0@lo:400:kworker.0 00000100:00100000:3.0:1787785101.811713:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668503680:0@lo:400:kworker.0 00000100:00100000:3.0:1787785101.811743:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785101.811768:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787785101.811774:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668503552:0@lo:400:kworker.0 00000100:00100000:3.0:1787785101.811800:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785101.811818:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668504064:0@lo:400:kworker.0 00000100:00100000:3.0:1787785101.811838:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785101.811907:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503808 00000100:00100000:2.0:1787785101.811911:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503808 00000100:00100000:2.0:1787785101.811947:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503680 00000100:00100000:2.0:1787785101.811949:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503680 00000100:00100000:2.0:1787785101.811966:0:20438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504064 00000100:00100000:2.0:1787785101.811969:0:20438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504064 00000100:00100000:2.0:1787785101.811982:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 690 00000100:00100000:2.0:1787785101.811988:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668503808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785101.812024:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3277:x1874627668503808:12345-0@lo:400:kworker.0 Request processed in 36us (677us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785101.812030:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 690 00000100:00100000:2.0:1787785101.812042:0:20438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 691 00000100:00100000:2.0:1787785101.812047:0:20438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668503680:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785101.812076:0:20438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539130b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3279:x1874627668503680:12345-0@lo:400:kworker.0 Request processed in 29us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785101.812225:0:20438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 691 00000100:00080000:3.0:1787785101.812269:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785101.812334:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503424 00000100:00100000:0.0:1787785101.812338:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503424 00000100:00100000:0.0:1787785101.812389:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:0.0:1787785101.812397:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539376940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668503424:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785101.812422:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785101.812459:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539376940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3276:x1874627668503424:12345-0@lo:13:osp-pre-1-0.0 Request processed in 64us (1497us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785101.812465:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:1.0:1787785101.816943:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 692 00000100:00100000:1.0:1787785101.816973:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668504064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785101.817064:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3279:x1874627668504064:12345-0@lo:400:kworker.0 Request processed in 106us (5224us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785101.817072:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 692 00000100:00080000:1.0:1787785101.817139:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787785101.817215:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894021ac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3276:1874627668503424:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787785101.817262:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3279:1874627668504064:0@lo:400:kworker.0 00000100:00100000:1.0:1787785101.817275:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3279:1874627668503680:0@lo:400:kworker.0 00000100:00100000:1.0:1787785101.817312:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3277:1874627668503808:0@lo:400:kworker.0 00000100:00100000:1.0:1787785101.817336:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668503552 00000100:00100000:1.0:1787785101.817342:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668503552 00000100:00100000:1.0:1787785101.817395:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:1.0:1787785101.817402:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668503552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785101.817440:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a5a93c05-6249-4607-a205-909a326a8fd1+19:3279:x1874627668503552:12345-0@lo:400:kworker.0 Request processed in 39us (5639us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785101.817445:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00100000:1.0:1787785101.817477:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:a5a93c05-6249-4607-a205-909a326a8fd1:3279:1874627668503552:0@lo:400:kworker.0 00010000:00010000:3.0:1787785102.257338:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785103.005709:0:28191:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785103.005718:0:28191:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785103.007697:0:28191:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785103.007710:0:28191:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785103.112283:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785103.112326:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785103.112355:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785103.112388:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785103.112405:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785103.112424:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785103.112450:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785103.112478:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402048 00000100:00100000:0.0:1787785103.112486:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402048 00000100:00100000:0.0:1787785103.112539:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402560 00000100:00100000:0.0:1787785103.112542:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402560 00000100:00100000:0.0:1787785103.112561:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402176 00000100:00100000:0.0:1787785103.112564:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402176 00000100:00100000:0.0:1787785103.112576:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402432 00000100:00100000:0.0:1787785103.112579:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402432 00000100:00100000:0.0:1787785103.112594:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 678 00000100:00100000:0.0:1787785103.112603:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658402048:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785103.112669:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658402048:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 67us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785103.112676:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 678 00000100:00100000:0.0:1787785103.112696:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 679 00000100:00100000:0.0:1787785103.112702:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658402560:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785103.112725:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ade00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658402560:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 23us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785103.112730:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 679 00000100:00100000:0.0:1787785103.112741:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 680 00000100:00100000:0.0:1787785103.112747:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658402176:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785103.112767:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2357:x1874627658402176:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785103.112772:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 680 00000100:00100000:0.0:1787785103.112783:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 681 00000100:00100000:0.0:1787785103.112789:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658402432:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785103.112809:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2357:x1874627658402432:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 21us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785103.112816:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 681 00000100:00100000:0.0:1787785103.112834:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402304 00000100:00100000:0.0:1787785103.112837:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402304 00000100:00100000:0.0:1787785103.112862:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401920 00000100:00100000:0.0:1787785103.112865:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401920 00000100:00100000:0.0:1787785103.112885:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 693 00000100:00100000:1.0:1787785103.112983:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658401792 00000100:00100000:1.0:1787785103.112988:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658401792 00000100:00100000:1.0:1787785103.113016:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 697 00000100:00100000:1.0:1787785103.113023:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658401792:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785103.113049:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658401792:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (776us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785103.113054:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 697 00000100:00080000:1.0:1787785103.113101:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787785103.113106:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785103.113107:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787785103.113120:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 694 00000100:00100000:1.0:1787785103.113127:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2357:x1874627658401920:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785103.113169:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2357:x1874627658401920:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 43us (763us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785103.113174:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 694 00000100:00100000:0.0:1787785103.113381:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658402304:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785103.113427:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2357:x1874627658402304:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 535us (1070us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785103.113434:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 693 00010000:00010000:3.0:1787785103.281367:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1787785103.601491:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785103.601501:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785103.602067:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785103.602080:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1787785103.947572:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:2.0:1787785103.948210:0:5796:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402688 00000100:00100000:2.0:1787785103.948220:0:5796:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402688 00000100:00100000:2.0:1787785103.948295:0:5796:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 79 00000100:00100000:2.0:1787785103.948305:0:5796:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+54:2356:x1874627658402688:12345-192.168.203.43@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787785103.948321:0:5796:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 16 locks, starting at 0 00010000:00010000:2.0:1787785103.948399:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905e00/0xb0ce63137acc4081 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b05be expref: 54 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948420:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff989505542100) 00010000:00010000:2.0:1787785103.948432:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540655e00/0xb0ce63137acc3ef2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0539 expref: 53 pid: 5813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948442:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff989510520600) 00010000:00010000:2.0:1787785103.948451:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654400/0xb0ce63137acc3e82 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0516 expref: 52 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948459:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff98950a162300) 00010000:00010000:2.0:1787785103.948467:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010adc00/0xb0ce63137acc3d24 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b049f expref: 51 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948487:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff98950a649b00) 00010000:00010000:2.0:1787785103.948496:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904e00/0xb0ce63137acc3cb4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b047c expref: 50 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948505:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff989519d44f00) 00010000:00010000:2.0:1787785103.948513:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9894010ad400/0xb0ce63137acc3b95 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b041a expref: 49 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948521:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff98950a649400) 00010000:00010000:2.0:1787785103.948528:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135a00/0xb0ce63137acc3b4f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0405 expref: 48 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948536:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x0 (ffff98950a649e00) 00010000:00010000:2.0:1787785103.948559:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508905800/0xb0ce63137acc3433 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x16:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b021b expref: 47 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948568:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x16:0x0].0x0 (ffff989510520e00) 00010000:00010000:2.0:1787785103.948575:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989508904a00/0xb0ce63137acc3193 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b015e expref: 46 pid: 10850 timeout: 445 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948583:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff989510520200) 00010000:00010000:2.0:1787785103.948590:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989401099400/0xb0ce63137acc31b6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x12:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b0165 expref: 45 pid: 5811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948598:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff98950a38c600) 00010000:00010000:2.0:1787785103.948604:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989403350600/0xb0ce63137acc2d72 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xf:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841b005b expref: 44 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948613:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff98950a38ca00) 00010000:00010000:2.0:1787785103.948640:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3800/0xb0ce63137acc2a93 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x51/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841aff89 expref: 43 pid: 10850 timeout: 421 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.948651:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff989519d44900) 00010000:00010000:2.0:1787785103.948667:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953f5b3200/0xb0ce63137acc2ab6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841aff90 expref: 42 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.949075:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc:0x0].0x0 (ffff989519d44e00) 00010000:00010000:2.0:1787785103.949095:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989540654c00/0xb0ce63137acc25e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afe5c expref: 41 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.949105:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff989510520500) 00010000:00010000:2.0:1787785103.949116:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff98953efb3200/0xb0ce63137acc21da lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afd28 expref: 40 pid: 10850 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.949125:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff989531111c00) 00010000:00010000:2.0:1787785103.949135:0:5796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff989539135c00/0xb0ce63137acc21b0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.43@tcp remote: 0xda2ac8b3841afd1a expref: 39 pid: 5812 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787785103.949143:0:5796:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff98950a38cb00) 00010000:00010000:2.0:1787785103.949243:0:5796:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787785103.949304:0:5796:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894021aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2356:x1874627658402688:12345-192.168.203.43@tcp:103:ldlm_bl.0 Request processed in 999us (1742us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785103.949313:0:5796:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 79 00010000:00010000:1.0:1787785104.308395:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785104.864273:0:28242:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785104.864282:0:28242:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785104.865288:0:28242:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785104.865300:0:28242:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1787785104.887690:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668504192:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787785104.887832:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785104.888077:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504192 00000100:00100000:2.0:1787785104.888085:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504192 00000100:00100000:2.0:1787785104.888173:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:2.0:1787785104.888183:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668504192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785104.888219:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785104.888268:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3279:x1874627668504192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (437us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785104.888276:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00080000:1.0:1787785104.888338:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787785104.888388:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668504192:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787785105.330361:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787785106.354394:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785106.851059:0:28293:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785106.851072:0:28293:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785106.851951:0:28293:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785106.851969:0:28293:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1787785106.930196:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668504320:0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930196:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530507480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668504448:0@lo:400:kworker.0 00000100:00100000:3.0:1787785106.930303:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787785106.930374:0:3276:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530505a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668504832:0@lo:400:kworker.0 00000100:00100000:3.0:1787785106.930407:0:3276:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785106.930521:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504832 00000100:00100000:2.0:1787785106.930531:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504832 00000100:00100000:1.0:1787785106.930585:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785106.930606:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530506d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668504960:0@lo:400:kworker.0 00000100:00100000:2.0:1787785106.930607:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504448 00000100:00100000:2.0:1787785106.930609:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504448 00000100:00100000:2.0:1787785106.930632:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 695 00000100:00100000:1.0:1787785106.930633:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785106.930642:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668504832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930663:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9895305043c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668504576:0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930688:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785106.930703:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98953937ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3276:x1874627668504832:12345-0@lo:400:kworker.0 Request processed in 62us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785106.930703:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668505088:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785106.930711:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 695 00000100:00100000:1.0:1787785106.930726:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785106.930729:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504576 00000100:00100000:2.0:1787785106.930732:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504576 00000100:00100000:1.0:1787785106.930746:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787785106.930748:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:1.0:1787785106.930752:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668504704:0@lo:400:kworker.0 00000100:00100000:2.0:1787785106.930754:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668504448:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930776:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785106.930783:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668504448:12345-0@lo:400:kworker.0 Request processed in 29us (198us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785106.930789:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00100000:2.0:1787785106.930799:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 697 00000100:00100000:2.0:1787785106.930806:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668504576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930807:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504320 00000100:00100000:1.0:1787785106.930811:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504320 00000100:00100000:2.0:1787785106.930838:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668504576:12345-0@lo:400:kworker.0 Request processed in 33us (149us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785106.930839:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 698 00000100:00100000:2.0:1787785106.930844:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 697 00000100:00100000:1.0:1787785106.930845:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668504320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930874:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:a5a93c05-6249-4607-a205-909a326a8fd1+19:3276:x1874627668504320:12345-0@lo:400:kworker.0 Request processed in 29us (572us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785106.930879:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 698 00000100:00100000:2.0:1787785106.930882:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9895305043c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668504576:0@lo:400:kworker.0 00000100:00100000:2.0:1787785106.930911:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530507480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668504448:0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930918:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504960 00000100:00100000:1.0:1787785106.930921:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504960 00000100:00100000:2.0:1787785106.930933:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530504780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a5a93c05-6249-4607-a205-909a326a8fd1:3276:1874627668504320:0@lo:400:kworker.0 00000100:00100000:2.0:1787785106.930943:0:3276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530505a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3276:1874627668504832:0@lo:400:kworker.0 00000100:00100000:1.0:1787785106.930946:0:10688:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668504704 00000100:00100000:1.0:1787785106.930948:0:10688:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668504704 00000100:00100000:1.0:1787785106.930973:0:10688:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1787785106.930980:0:10688:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668504960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785106.930981:0:6669:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00000100:00100000:2.0:1787785106.930987:0:6669:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668504704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785106.931017:0:6669:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668504704:12345-0@lo:400:kworker.0 Request processed in 30us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785106.931017:0:10688:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668504960:12345-0@lo:400:kworker.0 Request processed in 37us (382us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785106.931023:0:10688:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1787785106.931045:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505088 00000100:00100000:1.0:1787785106.931048:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505088 00000100:00080000:3.0:1787785106.931060:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787785106.931065:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787785106.931068:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787785106.931073:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1787785106.931079:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668505088:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785106.931101:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530506580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668504704:0@lo:400:kworker.0 00002000:00000024:1.0:1787785106.931110:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787785106.931236:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305052c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668505088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 157us (507us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785106.931244:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1787785106.931283:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989530506d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668504960:0@lo:400:kworker.0 00000100:00100000:3.0:1787785106.935555:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668505088:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785106.936770:0:6669:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00010000:00010000:1.0:1787785107.377647:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787785108.233722:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785108.233771:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785108.233807:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785108.233840:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785108.233864:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785108.233889:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785108.233898:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403328 00000100:00100000:1.0:1787785108.233915:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:0.0:1787785108.233918:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403328 00000100:00100000:1.0:1787785108.233939:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403072 00000100:00100000:1.0:1787785108.233945:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403072 00000100:00100000:0.0:1787785108.233984:0:5813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402944 00000100:00100000:1.0:1787785108.233985:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403584 00000100:00100000:0.0:1787785108.233989:0:5813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402944 00000100:00100000:1.0:1787785108.233990:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403584 00000100:00100000:1.0:1787785108.234021:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403200 00000100:00100000:1.0:1787785108.234024:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403200 00000100:00100000:0.0:1787785108.234030:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 698 00000100:00100000:1.0:1787785108.234043:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403456 00000100:00100000:0.0:1787785108.234043:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2357:x1874627658403328:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785108.234048:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403456 00000100:00100000:1.0:1787785108.234067:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 684 00000100:00100000:1.0:1787785108.234075:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658403072:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:0.0:1787785108.234125:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2357:x1874627658403328:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 85us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785108.234137:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 698 00000100:00100000:2.0:1787785108.234182:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658402816 00000100:00100000:0.0:1787785108.234187:0:5813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 699 00000100:00100000:2.0:1787785108.234188:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658402816 00000100:00100000:0.0:1787785108.234197:0:5813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658402944:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785108.234224:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 699 00000100:00100000:2.0:1787785108.234241:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658402816:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:3.0:1787785108.234270:0:13825:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 685 00000100:00100000:3.0:1787785108.234281:0:13825:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989527de8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658403584:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785108.239555:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2357:x1874627658403072:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 5469us (5771us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785108.239571:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 684 00000100:00100000:1.0:1787785108.239610:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 686 00000100:00100000:1.0:1787785108.239623:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658403200:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785108.239670:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658403200:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 52us (5782us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785108.239676:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 686 00000100:00100000:1.0:1787785108.239688:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 687 00000100:00100000:1.0:1787785108.239696:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658403456:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785108.239723:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658403456:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 28us (5808us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785108.239729:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 687 00000100:00080000:1.0:1787785108.240303:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785108.240308:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787785108.240311:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787785108.244135:0:5813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658402944:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 9926us (10256us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785108.244169:0:5813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 699 00000100:00100000:2.0:1787785108.244961:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2357:x1874627658402816:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 10716us (11244us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785108.244971:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 699 00000100:00100000:3.0:1787785108.245597:0:13825:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989527de8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2357:x1874627658403584:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 11314us (11751us total) trans 0 rc 0/0 00000100:00100000:3.0:1787785108.245608:0:13825:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 685 00010000:00010000:3.0:1787785108.401624:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1787785108.854096:0:28344:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1787785108.854105:0:28344:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1787785108.855608:0:28344:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1787785108.855621:0:28344:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785109.425822:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785110.001873:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787785110.002693:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668505216:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787785110.002761:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785110.002921:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505216 00000100:00100000:1.0:1787785110.002929:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505216 00000100:00100000:1.0:1787785110.003747:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:1.0:1787785110.003759:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989539379e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668505216:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787785110.003808:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:1.0:1787785110.003869:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989539379e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668505216:12345-0@lo:13:osp-pre-0-0.0 Request processed in 113us (1107us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785110.003877:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:0.0:1787785110.004093:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668505216:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1787785110.005222:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787785110.449745:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1787785110.685504:0:28398:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1787785110.685515:0:28398:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1787785110.688853:0:28398:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1787785110.688872:0:28398:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785111.473474:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787785112.051122:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668505600:0@lo:400:kworker.0 00000100:00100000:3.0:1787785112.051211:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785112.051256:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98953937b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668505472:0@lo:400:kworker.0 00000100:00100000:3.0:1787785112.051276:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787785112.051285:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668505728:0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.051304:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785112.051327:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668505984:0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.051352:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785112.051372:0:3277:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1787785112.051377:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff989539378780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668505344:0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.051411:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785112.051449:0:3277:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668505856:0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.051472:0:3277:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787785112.051501:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505600 00000100:00100000:1.0:1787785112.051508:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505600 00000100:00100000:1.0:1787785112.051554:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505472 00000100:00100000:1.0:1787785112.051558:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505472 00000100:00100000:1.0:1787785112.051576:0:5811:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505856 00000100:00100000:1.0:1787785112.051579:0:5811:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505856 00000100:00100000:1.0:1787785112.051595:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 700 00000100:00100000:1.0:1787785112.051614:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668505600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.051658:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3278:x1874627668505600:12345-0@lo:400:kworker.0 Request processed in 46us (447us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785112.051665:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 700 00000100:00100000:1.0:1787785112.051685:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 701 00000100:00100000:1.0:1787785112.051691:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668505472:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.051734:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530507480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3277:x1874627668505472:12345-0@lo:400:kworker.0 Request processed in 43us (442us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785112.051740:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 701 00000100:00100000:1.0:1787785112.051751:0:5811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 702 00000100:00100000:1.0:1787785112.051757:0:5811:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668505856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.051785:0:5811:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530505e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3277:x1874627668505856:12345-0@lo:400:kworker.0 Request processed in 28us (312us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785112.051790:0:5811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 702 00000100:00100000:1.0:1787785112.051809:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505984 00000100:00100000:1.0:1787785112.051812:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505984 00000100:00100000:2.0:1787785112.052279:0:3279:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787785112.052291:0:3279:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668506112:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787785112.052336:0:3279:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1787785112.052395:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787785112.052400:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787785112.052415:0:5803:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505344 00000100:00100000:2.0:1787785112.052419:0:5803:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505344 00000100:00100000:2.0:1787785112.052449:0:5803:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 700 00000100:00100000:2.0:1787785112.052456:0:5803:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668505344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787785112.052490:0:5803:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:a5a93c05-6249-4607-a205-909a326a8fd1+19:3277:x1874627668505344:12345-0@lo:400:kworker.0 Request processed in 34us (1077us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785112.052497:0:5803:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 700 00000100:00100000:2.0:1787785112.052559:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff989539378780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a5a93c05-6249-4607-a205-909a326a8fd1:3277:1874627668505344:0@lo:400:kworker.0 00000100:00100000:2.0:1787785112.052572:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3277:1874627668505856:0@lo:400:kworker.0 00000100:00100000:2.0:1787785112.052582:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98953937b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3277:1874627668505472:0@lo:400:kworker.0 00000100:00100000:3.0:1787785112.052935:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787785112.053007:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668506112 00000100:00100000:0.0:1787785112.053012:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668506112 00000100:00100000:0.0:1787785112.053044:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:0.0:1787785112.053052:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668506112:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787785112.053082:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100335616 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1787785112.053126:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98954028c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3279:x1874627668506112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (789us total) trans 0 rc 0/0 00000100:00100000:0.0:1787785112.053132:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:0.0:1787785112.053200:0:3279:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3279:1874627668506112:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787785112.056113:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3278:1874627668505600:0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.061046:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668505728 00000100:00100000:1.0:1787785112.061062:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668505728 00000100:00100000:1.0:1787785112.061699:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 688 00000100:00100000:1.0:1787785112.061729:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668505984:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.061826:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895305043c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3277:x1874627668505984:12345-0@lo:400:kworker.0 Request processed in 116us (10471us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785112.061851:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 688 00000100:00100000:1.0:1787785112.061869:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 689 00000100:00100000:1.0:1787785112.061876:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668505728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.061913:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9894035d9680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668505728:12345-0@lo:400:kworker.0 Request processed in 39us (8980us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785112.061920:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 689 00000100:00080000:1.0:1787785112.061941:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787785112.061977:0:3277:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3277:1874627668505984:0@lo:400:kworker.0 00000100:00100000:1.0:1787785112.062015:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9894035d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668505728:0@lo:400:kworker.0 00000040:00080000:1.0:1787785112.408853:0:28450:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1787785112.408862:0:28450:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1787785112.409555:0:28450:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1787785112.409572:0:28450:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785112.502073:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787785113.352689:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785113.352732:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785113.352764:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785113.352804:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785113.352822:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785113.352849:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785113.352869:0:3269:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.43@tcp (source: 12345-192.168.203.43@tcp) 00000100:00100000:1.0:1787785113.352899:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403968 00000100:00100000:1.0:1787785113.352908:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403968 00000100:00100000:1.0:1787785113.352960:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658404480 00000100:00100000:1.0:1787785113.352963:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658404480 00000100:00100000:1.0:1787785113.352978:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658404096 00000100:00100000:1.0:1787785113.352980:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658404096 00000100:00100000:1.0:1787785113.352990:0:6671:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658404352 00000100:00100000:1.0:1787785113.352993:0:6671:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658404352 00000100:00100000:1.0:1787785113.353007:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 690 00000100:00100000:1.0:1787785113.353221:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658403968:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785113.353306:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530504b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+17:2356:x1874627658403968:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 90us (572us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785113.353313:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 690 00000100:00100000:1.0:1787785113.353329:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 691 00000100:00100000:1.0:1787785113.353336:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658404480:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785113.353360:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+12:2356:x1874627658404480:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 24us (565us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785113.353366:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 691 00000100:00100000:1.0:1787785113.353377:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 692 00000100:00100000:1.0:1787785113.353383:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658404096:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785113.353405:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b5352c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+38:2356:x1874627658404096:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785113.353411:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 692 00000100:00100000:1.0:1787785113.353422:0:6671:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 693 00000100:00100000:1.0:1787785113.353428:0:6671:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658404352:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:1.0:1787785113.353449:0:6671:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b534b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:5882e86d-aef6-465c-ae01-bd8a61753eef+9:2356:x1874627658404352:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 22us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1787785113.353455:0:6671:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 693 00000100:00080000:1.0:1787785113.353472:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787785113.353476:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787785113.353835:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403712 00000100:00100000:2.0:1787785113.353840:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403712 00000100:00100000:2.0:1787785113.353876:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 701 00000100:00100000:2.0:1787785113.353884:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658403712:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785113.353919:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9895064e3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5f65ba5e-56c4-467c-9aa4-fe366ef1d7aa+13:2356:x1874627658403712:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 36us (1241us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785113.353925:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 701 00000100:00100000:2.0:1787785113.353953:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658404224 00000100:00100000:2.0:1787785113.353956:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658404224 00000100:00100000:2.0:1787785113.353992:0:10850:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627658403840 00000100:00100000:2.0:1787785113.353995:0:10850:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627658403840 00000100:00100000:2.0:1787785113.354013:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 703 00000100:00100000:2.0:1787785113.354019:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2356:x1874627658404224:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785113.354044:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff989530506940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:5882e86d-aef6-465c-ae01-bd8a61753eef+38:2356:x1874627658404224:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 27us (1281us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785113.354051:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 703 00000100:00100000:2.0:1787785113.354061:0:10850:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.43@tcp, seq: 704 00000100:00100000:2.0:1787785113.354066:0:10850:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658403840:12345-192.168.203.43@tcp:400:kworker.0 00000100:00100000:2.0:1787785113.354086:0:10850:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b535a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:3aa20aaf-dc32-4dd7-b302-1a1d759b54b2+95:2356:x1874627658403840:12345-192.168.203.43@tcp:400:kworker.0 Request processed in 20us (1264us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785113.354092:0:10850:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.43@tcp, seq: 704 00000100:00080000:2.0:1787785113.354174:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787785113.523851:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1787785113.841406:0:11668:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1787785113.841415:0:11668:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1787785113.842245:0:11668:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1787785113.842256:0:11668:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1787785114.545392:0:5799:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787785115.123016:0:28500:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12u: @@@@@@ FAIL: request on 0x200000402:0x35:0x0 is not SUCCEED on mds1 00000100:00100000:1.0:1787785115.123302:0:3278:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787785115.123325:0:3278:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668506240:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787785115.123391:0:3278:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787785115.123499:0:6672:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874627668506240 00000100:00100000:2.0:1787785115.123507:0:6672:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874627668506240 00000100:00100000:2.0:1787785115.123571:0:6672:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1787785115.123581:0:6672:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668506240:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787785115.123615:0:6672:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 102039552 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1787785115.123667:0:6672:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff98951b536940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3278:x1874627668506240:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1787785115.123675:0:6672:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1787785115.123721:0:3278:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98954028c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3278:1874627668506240:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787785115.123767:0:5806:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787785115.574361:0:17918:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787785116.604563:0:5798:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 68796 lines, 68796 kept, 0 dropped, 0 bad.